tar4ibd too small read block size for free space on ibdata

MySQL & MariaDB Percona XtraBackup
1

Description:

tar4ibd make too many small block read for free .ibd files. I am xtrabacking on slave database,but the query sql request on slave will much slower while dump ibdata2. Is there some way to reslove it?

for ibdata1,which have data on it,16k block request to os. this is correct.

tar4ibd -d -c ibdata1 |gzip > /bak/a.tar.gz

tar4ibd: using O_DIRECT for the input file.
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 6.50 0.00 1672.00 0.00 26640.00 0.00 31.87 0.11 0.06 0.06 10.75

12:55:37 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:55:38 PM 18 11.88 0.00 2.97 13.86 0.00 0.99 0.00 70.30 1678.00

for ibdata2,maybe no data on it. you will see 24000 seq read iops to os,which block size is 1kb at all.cpu iowait much higher.
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 3.00 24281.00 2.50 24281.00 22.00 2.00 0.82 0.04 0.03 67.80

12:55:10 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:55:11 PM 18 5.05 0.00 40.40 41.41 5.05 8.08 0.00 0.00 24763.00

REPLAY:
dd if=/dev/zero of=a.z count=10000 bs=1024k
strace tar4ibd -d -c a.z|gzip > /bak/a.tar.gz
and you will see
read(3, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 1024) = 1024
write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 512) = 512
write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 512) = 512
read(3, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 1024) = 1024
write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 512) = 512
write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 512) = 512
read(3, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0 "…, 1024) = 1024

2

modified page_size from 11024 to 81024 on function tar_append_regfile(TAR *t, char realname),
which is on tar4ibd_libtar-1.2.11.patch line 396
or
on libtar-1.2.11/lib/append.c line 539。
539c539
< int page_size = 1 * 1024; / check 1K, 2K, 4K, 8K and 16K */

int page_size = 8 * 1024; /* check 1K, 2K, 4K, 8K and 16K */

after that,the block size will change from 1k to 8k，it will help to my issue,but i don’t konw if it will hit bugs?

3

Right, tar4ibd reads file inefficiently.

For compressed tablespaces, tar4ibd has to start with the smallest possible block size (i.e. 1KB) to verify page integrity. However, it can be optimized by:

  • reading in large chunks and then processing it in smaller ones;
  • using default page size (16KB) for uncompressed files

In other words, do what xtrabackup binary does when reading the input files.

I have reported [URL]https://bugs.launchpad.net/percona-xtrabackup/+bug/899931[/URL] so this issue is on our radar. However, we are going to remove tar4ibd (and use xtrabackup binary for streaming) in the next major version. So it’s unlikely we will invest any efforts into fixing this in 1.6.