[Xtrabackup] Database page corruption detected

MySQL & MariaDB Percona XtraBackup
1

Hello everyone,

I have been using xtrabackup for backups for a long time. Until now, I have never had any problems. However, as of today, I started receiving the following error:

root@clone:~# /usr/bin/xtrabackup --backup --target-dir=/mysql-backup/backup2
2024-12-24T22:57:01.369128+03:00 0 [Note] [MY-011825] [Xtrabackup] recognized server arguments: --server-id=1 --datadir=/var/lib/mysql --innodb_buffer_pool_size=35G --innodb_log_file_size=1G --innodb_flush_log_at_trx_commit=2
2024-12-24T22:57:01.369465+03:00 0 [Note] [MY-011825] [Xtrabackup] recognized client arguments: --socket=/var/run/mysqld/mysqld.sock --backup=1 --target-dir=/mysql-backup/backup2
/usr/bin/xtrabackup version 8.0.35-31 based on MySQL server 8.0.35 Linux (x86_64) (revision id: 55ec21d7)
241224 22:57:01  version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup;mysql_socket=/var/run/mysqld/mysqld.sock' (using password: NO).
241224 22:57:01  version_check Connected to MySQL server
241224 22:57:01  version_check Executing a version check against the server...
241224 22:57:01  version_check Done.
2024-12-24T22:57:01.548621+03:00 0 [Note] [MY-011825] [Xtrabackup] Connecting to MySQL server host: localhost, user: not set, password: not set, port: not set, socket: /var/run/mysqld/mysqld.sock
2024-12-24T22:57:01.572284+03:00 0 [Note] [MY-011825] [Xtrabackup] Using server version 8.0.34-26.1
2024-12-24T22:57:01.577205+03:00 0 [Note] [MY-011825] [Xtrabackup] Executing LOCK TABLES FOR BACKUP ...
2024-12-24T22:57:01.579608+03:00 0 [Note] [MY-011825] [Xtrabackup] uses posix_fadvise().
2024-12-24T22:57:01.579671+03:00 0 [Note] [MY-011825] [Xtrabackup] cd to /var/lib/mysql
2024-12-24T22:57:01.579703+03:00 0 [Note] [MY-011825] [Xtrabackup] open files limit requested 0, set to 1024
2024-12-24T22:57:01.854507+03:00 0 [Note] [MY-011825] [Xtrabackup] using the following InnoDB configuration:
2024-12-24T22:57:01.854566+03:00 0 [Note] [MY-011825] [Xtrabackup] innodb_data_home_dir = .
2024-12-24T22:57:01.854576+03:00 0 [Note] [MY-011825] [Xtrabackup] innodb_data_file_path = ibdata1:12M:autoextend
2024-12-24T22:57:01.854608+03:00 0 [Note] [MY-011825] [Xtrabackup] innodb_log_group_home_dir = ./
2024-12-24T22:57:01.854625+03:00 0 [Note] [MY-011825] [Xtrabackup] innodb_log_files_in_group = 2
2024-12-24T22:57:01.854649+03:00 0 [Note] [MY-011825] [Xtrabackup] innodb_log_file_size = 1073741824
2024-12-24T22:57:01.858288+03:00 0 [Note] [MY-011825] [Xtrabackup] inititialize_service_handles suceeded
2024-12-24T22:57:01.936665+03:00 0 [Note] [MY-011825] [Xtrabackup] Connecting to MySQL server host: localhost, user: not set, password: not set, port: not set, socket: /var/run/mysqld/mysqld.sock
2024-12-24T22:57:01.956663+03:00 0 [Note] [MY-011825] [Xtrabackup] Redo Log Archiving is not set up.
2024-12-24T22:57:02.038157+03:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (2358094913482)
2024-12-24T22:57:02.057343+03:00 0 [Note] [MY-012953] [InnoDB] Disabling background ibuf IO read threads.
2024-12-24T22:57:02.259169+03:00 0 [Note] [MY-011825] [Xtrabackup] Generating a list of tablespaces
2024-12-24T22:57:02.259316+03:00 0 [Note] [MY-012204] [InnoDB] Scanning './'
2024-12-24T22:57:02.262843+03:00 0 [Note] [MY-012208] [InnoDB] Completed space ID check of 2 files.
2024-12-24T22:57:02.263100+03:00 0 [Warning] [MY-012091] [InnoDB] Allocated tablespace ID 4 for mysql/wsrep_streaming_log, old maximum was 0
2024-12-24T22:57:02.276989+03:00 0 [Note] [MY-013252] [InnoDB] Using undo tablespace './undo_001'.
2024-12-24T22:57:02.278321+03:00 0 [Note] [MY-013252] [InnoDB] Using undo tablespace './undo_002'.
2024-12-24T22:57:02.280341+03:00 0 [Note] [MY-012910] [InnoDB] Opened 2 existing undo tablespaces.
2024-12-24T22:57:02.281173+03:00 2 [Note] [MY-011825] [Xtrabackup] Copying ./dbname/content_counters.ibd to /mysql-backup/backup2/dbname/content_counters.ibd
2024-12-24T22:57:03.038487+03:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (2358094913482)
2024-12-24T22:57:04.038785+03:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (2358094913482)
2024-12-24T22:57:05.039172+03:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (2358094913482)
2024-12-24T22:57:05.828965+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:05.935684+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.039524+03:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (2358094913482)
2024-12-24T22:57:06.044288+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.150121+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.256036+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.362218+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.468329+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.574079+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.679693+03:00 2 [Note] [MY-011825] [Xtrabackup] Database page corruption detected at page 41216, retrying...
2024-12-24T22:57:06.785944+03:00 2 [ERROR] [MY-011825] [Xtrabackup] failed to read page after 10 retries. File ./dbname/content_counters.ibd seems to be corrupted.
2024-12-24T22:57:06.803313+03:00 2 [ERROR] [MY-011825] [Xtrabackup] xtrabackup_copy_datafile() failed
2024-12-24T22:57:06.803375+03:00 2 [ERROR] [MY-011825] [Xtrabackup] failed to copy datafile ./dbname/content_counters.ibd
2024-12-24T22:57:07.039806+03:00 1 [Note] [MY-011825] [Xtrabackup] >> log scanned up to (2358094913482)```

root@clone:~# mysql --version
mysql  Ver 8.0.34-26.1 for Linux on x86_64 (Percona XtraDB Cluster (GPL), Release rel26, Revision 0988772, WSREP version 26.1.4.3)
root@clone:~# xtrabackup --version
2024-12-24T22:58:24.532706+03:00 0 [Note] [MY-011825] [Xtrabackup] recognized server arguments: --server-id=1 --datadir=/var/lib/mysql --innodb_buffer_pool_size=35G --innodb_log_file_size=1G --innodb_flush_log_at_trx_commit=2
xtrabackup version 8.0.35-31 based on MySQL server 8.0.35 Linux (x86_64) (revision id: 55ec21d7)```

mysql> CHECK TABLE dbname.content_counters;
+----------------------------+-------+----------+----------+
| Table                      | Op    | Msg_type | Msg_text |
+----------------------------+-------+----------+----------+
| dbname.content_counters | check | status   | OK       |
+----------------------------+-------+----------+----------+
1 row in set (34.03 sec)```

Thanks Percona Team

2

Hi, it’s possible that the problem is in the header of the page, which CHECK TABLE ignores for Innodb tables. You should probably run innochecksum to verify but that requires shutting down the server. I would suggest a full table rebuild using either ALTER TABLE FORCE or pt-online-schema-change.