Hi,
We have a cluster with 3 nodes on centos 6.5. This is the version of the application :
Percona-XtraDB-Cluster-server-56-5.6.20-25.7.888.el6.x86_64
percona-toolkit-2.2.10-1.noarch
percona-xtrabackup-2.2.3-4982.el6.x86_64
Percona-XtraDB-Cluster-galera-3-3.7-1.3254.rhel6.x86_64
Percona-XtraDB-Cluster-galera-3-debuginfo-3.7-1.3254.rhel6.x86_64
percona-release-0.0-1.x86_64
Percona-XtraDB-Cluster-test-56-5.6.20-25.7.888.el6.x86_64
Percona-XtraDB-Cluster-shared-56-5.6.20-25.7.888.el6.x86_64
Percona-XtraDB-Cluster-full-56-5.6.20-25.7.888.el6.x86_64
percona-nagios-plugins-1.1.4-1.noarch
Percona-Server-shared-compat-5.1.68-rel14.6.551.rhel6.x86_64
Percona-XtraDB-Cluster-56-debuginfo-5.6.20-25.7.888.el6.x86_64
Percona-XtraDB-Cluster-client-56-5.6.20-25.7.888.el6.x86_64
Percona-XtraDB-Cluster-garbd-3-3.7-1.3254.rhel6.x86_64
We execute some alter table on the first node. The slave nodes crash with the error :
After this, all the backup do with innobackupex are corrupt.
When we do innobackupex --apply-log --export . we have the error :
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
InnoDB: Using atomics to ref count buffer pool pages
InnoDB: The InnoDB memory heap is disabled
InnoDB: Mutexes and rw_locks use GCC atomic builtins
InnoDB: Compressed tables use zlib 1.2.3
InnoDB: Using CPU crc32 instructions
InnoDB: Initializing buffer pool, size = 100.0M
InnoDB: Completed initialization of buffer pool
InnoDB: Highest supported file format is Barracuda.
InnoDB: The log sequence numbers 146601703753 and 146601703753 in ibdata files do not match the log sequence number 158456216210 in the ib_logfiles!
InnoDB: Database was not shutdown normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files…
InnoDB: Restoring possible half-written data pages
InnoDB: from the doublewrite buffer…
InnoDB: Last MySQL binlog file position 0 19917434, file name t2-mysql01-bin.000029
InnoDB: 128 rollback segment(s) are active.
InnoDB: Waiting for purge to start
InnoDB: Page directory corruption: infimum not pointed to
2014-09-26 10:15:31 7f23bd7a6700 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex deadbeef000000000000000000000000000000000000000000 00000000000000000000000000434f4d50414354
…
…
;
InnoDB: End of page dump
2014-09-26 10:15:31 7f23bd7a6700 InnoDB: uncompressed page, stored checksum in field1 3735928559, calculated checksums for field1: crc32 177807968, innodb 3475755373, none 3735928559, stored checksum in field2 3735928559, calculated checksums for field2: crc32 177807968, innodb 1624572976, none 3735928559, page LSN 0 0, low 4 bytes of LSN at page end 0, page number (if stored to page already) 0, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
InnoDB: Page directory corruption: supremum not pointed to
2014-09-26 10:15:31 7f23bd7a6700 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex deadbeef000000000000000000000000000000000000000000 00000000000000000000000000434f4d504143545
…
…
…
;
InnoDB: End of page dump
2014-09-26 10:15:31 7f23bd7a6700 InnoDB: uncompressed page, stored checksum in field1 3735928559, calculated checksums for field1: crc32 177807968, innodb 3475755373, none 3735928559, stored checksum in field2 3735928559, calculated checksums for field2: crc32 177807968, innodb 1624572976, none 3735928559, page LSN 0 0, low 4 bytes of LSN at page end 0, page number (if stored to page already) 0, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
innobackupex: Error:
innobackupex: ibbackup failed at /usr/bin/innobackupex line 2610.
When i add the line innodb_force_recovery = 2 in the backup-my.cnf it work.
Can you explain the source of the error. How can i correct the build of backup.
Thank’s