Not the answer you need?
Register and ask your own question!

--apply-log sporadic failures

micemice EntrantCurrent User Role Beginner
Hi all!
Trying to fight sporadic restore failures. I'm backuping about 200 mysql instances from slave replicas, average of 6 slave instances per server. Script is simply runs innobackupex over instances one by one.
Backup string:


After all backups created, on backup server I starting backup consistency checks by simply extracting backup and applying logs:
pbzip2 -vdc $file | xbstream -v -x -C /backup/checkdir/ && cd /backup/checkdir/
innobackupex --ibbackup=xtrabackup_55 --apply-log /backup/checkdir/

And sometime I got the following errors applying log:



























innobackupex: ibbackup failed at /usr/local/bin/innobackupex line 2560.

OR

InnoDB: Doing recovery: scanned up to log sequence number 4988629964288 (61 %)
InnoDB: Doing recovery: scanned up to log sequence number 4988635207168 (62 %)
140510 13:20:18 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 140510 13:20:19 InnoDB: Assertion failure in thread 140548991547136 in file rem0rec.c line 569
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
innobackupex: Error:
innobackupex: ibbackup failed at /usr/local/bin/innobackupex line 2560.

OR

InnoDB: Doing recovery: scanned up to log sequence number 4827916240896 (30 %)
InnoDB: Doing recovery: scanned up to log sequence number 4827921483776 (37 %)
140509 11:23:17 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 InnoDB: Probable data corruption on page
2307494
InnoDB: Original record (compact record)
InnoDB: on that page.
InnoDB: Cannot find the dir slot for record (compact record)
InnoDB: on that page!
140509 11:23:22 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex 000000 .... ---CUT---
...
InnoDB: End of page dump
140509 11:23:22 InnoDB: Page checksum 1244108629 (32bit_calc: 3907307010), prior-to-4.0.14-form checksum 2889030552
InnoDB: stored checksum 0, prior-to-4.0.14-form stored checksum 1124
InnoDB: Page lsn 1124 377661183, low 4 bytes of lsn at page end 377661183
InnoDB: Page number (if stored to page already) 2307494,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 204
InnoDB: Page may be an index page where index id is 523
140509 11:23:22 InnoDB: Assertion failure in thread 139719660447488 in file page0page.c line 153
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
innobackupex: Error:
innobackupex: ibbackup failed at /usr/local/bin/innobackupex line 2560.

There are no any regularity which backup fails: it could be 8 file broken or 0 from day to day, on different instances on different slave servers.
Using:
Percona mysqld Ver 5.5.16 for Linux on x86_64 (Source distribution)
xtrabackup 2.1.8 (also tested on many previous versions)
on Linux 2.6.39.2 x86_64

Comments

  • micemice Entrant Current User Role Beginner
    Updated xtrabackup to version 2.1.9. Still same problems.
  • micemice Entrant Current User Role Beginner
    No chance to get any support on this important meter, when I have data corruption, using xtrabackup?
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.