We are using MySQL Community Server 5.5.29 and percona-xtrabackup-2.2.5-5027.el6.x86_64
with InnoDB (innodb_file_per_table).
The backup is aborting after a few minutes with this error:
InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.
This software is published under
the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
Get the latest version of Percona XtraBackup, documentation, and help resources:
http://www.percona.com/xb/p
160119 13:10:43 innobackupex: Connecting to MySQL server with DSN ‘dbi:mysql:;mysql_read_default_group=xtrabackup’ as ‘root’ (using password: YES).
160119 13:10:43 innobackupex: Connected to MySQL server
160119 13:10:43 innobackupex: Executing a version check against the server…
160119 13:10:43 innobackupex: Done.
IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints “completed OK!”.
innobackupex: Using mysql server version 5.5.29
innobackupex: Created backup directory /data/mysqlbackup/2016-01-19_13-10-43
160119 13:10:43 innobackupex: Starting ibbackup with command:
xtrabackup
–defaults-group=“mysqld”
–backup
–suspend-at-end
–target-dir=/data/mysqlbackup/2016-01-19_13-10-43
–tmpdir=/var/lib/mysql/tmp
–use-memory=10G
–extra-lsndir=‘/var/lib/mysql/tmp’
innobackupex: Waiting for ibbackup (pid=25998) to suspend
innobackupex: Suspend file ‘/data/mysqlbackup/2016-01-19_13-10-43/xtrabackup_suspended_2’
xtrabackup version 2.2.5 based on MySQL server 5.6.21 Linux (x86_64) (revision id: )
xtrabackup: uses posix_fadvise().
xtrabackup: cd to /var/lib/mysql
xtrabackup: open files limit requested 102400, set to 102400
xtrabackup: using the following InnoDB configuration:
xtrabackup: innodb_data_home_dir = /var/lib/mysql/idbdata
xtrabackup: innodb_data_file_path = idbdata01:2G:autoextend
xtrabackup: innodb_log_group_home_dir = /var/lib/mysql/idblog
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 1073741824
xtrabackup: using O_DIRECT
log scanned up to (9868489235053)
[01] Copying /var/lib/mysql/idbdata/idbdata01 to /data/mysqlbackup/2016-01-19_13-10-43/idbdata01
log scanned up to (9868489235073)
log scanned up to (9868489235073)
log scanned up to (9868489235368)
log scanned up to (9868489235368)
log scanned up to (9868489235834)
log scanned up to (9868489235834)
log scanned up to (9868489235834)
log scanned up to (9868491216496)
log scanned up to (9868493327763)
log scanned up to (9868493608898)
log scanned up to (9868493645151)
log scanned up to (9868493646385)
log scanned up to (9868493647052)
log scanned up to (9868493647052)
log scanned up to (9868493647517)
log scanned up to (9868493647517)
log scanned up to (9868493647517)
log scanned up to (9868493647767)
log scanned up to (9868493647767)
log scanned up to (9868493649035)
log scanned up to (9868493649035)
log scanned up to (9868496525927)
log scanned up to (9868497839778)
log scanned up to (9868498830477)
log scanned up to (9868500494321)
log scanned up to (9868500494382)
log scanned up to (9868500494382)
log scanned up to (9868500494615)
log scanned up to (9868500494615)
log scanned up to (9868500495081)
log scanned up to (9868500495091)
log scanned up to (9868500495101)
log scanned up to (9868501246079)
log scanned up to (9868503846069)
log scanned up to (9868504726785)
log scanned up to (9868504726785)
log scanned up to (9868504798588)
log scanned up to (9868504799081)
log scanned up to (9868504799081)
log scanned up to (9868504799564)
log scanned up to (9868504799564)
log scanned up to (9868504799564)
log scanned up to (9868504799813)
log scanned up to (9868504799813)
log scanned up to (9868504800296)
log scanned up to (9868504800296)
log scanned up to (9868504800296)
log scanned up to (9868508666113)
log scanned up to (9868511227997)
log scanned up to (9868511228480)
log scanned up to (9868511228480)
log scanned up to (9868511228480)
log scanned up to (9868511228713)
log scanned up to (9868511228713)
log scanned up to (9868511229179)
log scanned up to (9868511229179)
log scanned up to (9868511229179)
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
log scanned up to (9868511229421)
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Database page corruption detected at page 206334, retrying…
[01] xtrabackup: Error: failed to read page after 10 retries. File /var/lib/mysql/idbdata/idbdata01 seems to be corrupted.
[01] xtrabackup: Error: xtrabackup_copy_datafile() failed.
[01] xtrabackup: Error: failed to copy datafile.
innobackupex: Error: The xtrabackup child process has died at /usr/bin/innobackupex line 2681.
innochecksum shows probelm in log sequence number check
[root@atfkmysql04 idbdata]# innochecksum -p 206334 /var/lib/mysql/idbdata/idbdata01 -v -d
file /var/lib/mysql/idbdata/idbdata01 = 3833593856 bytes (233984 pages)…
checking pages in range 206334 to 206334
page 206334: log sequence number: first = 2722042417; second = 0 <<<<<<<<<<<<<<<<<
page 206334 invalid (fails log sequence number check)
I am currently innodbchecksumming all ibd files, but so far no other files are affected. We are using innodb_file_per_table parameter, so there should not be user data in idb system tablespace.
How can I repair this corruption?
Best regards,
Martin