# Xtrabackup is failing once log is applied **URL:** \ **Category:** Percona XtraBackup\ **Created:** [April 17, 2013, 9:45am UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636 "2013-04-17T09:45:16Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![DmitryK](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@DmitryK](https://forums.percona.com/u/DmitryK)\ **Post date:** [April 17, 2013, 9:45am UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636/1 "2013-04-17T09:45:16Z")
MariaDB 5.5.29, Xtrabackup 2.0.5 on el5 - all from repo. Works fine on several servers with smaller datasets, but fails on the big one (350G): ``` InnoDB: Apply batch completed InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 995545447, file name mysql-bin.001309 InnoDB: and relay log file InnoDB: position 0 995545731, file name ./mysql-relay-bin.002034 InnoDB: Last MySQL binlog file position 0 276207520, file name ./mysql-bin.000005 130417 10:18:35 InnoDB: Waiting for the background threads to start 130417 10:18:36 Percona XtraDB ([url]http://www.percona.com[/url]) 1.1.8-20.1 started; log sequence number 13758730250258 130417 10:18:38 InnoDB: Assertion failure in thread 1191156032 in file xtrabackup.c line 1632 InnoDB: Failing assertion: cset == 0 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to [url]http://bugs.mysql.com[/url]. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: [url]http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html[/url] InnoDB: about forcing recovery. innobackupex: Error: innobackupex: ibbackup failed at /usr/bin/innobackupex line 381. Any suggestions what to check? Workarounds may be? Thanks! ``` ---
**Author:** ![gmouse](https://avatars.discourse-cdn.com/v4/letter/g/b9e5f3/32.png) [@gmouse](https://forums.percona.com/u/gmouse)\ **Post date:** [April 17, 2013, 2:22pm UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636/2 "2013-04-17T14:22:26Z")
Do you use special features of MariaDB that are incompatible with standard InnoDB? ---
**Author:** ![DmitryK](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@DmitryK](https://forums.percona.com/u/DmitryK)\ **Post date:** [April 18, 2013, 8:19am UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636/3 "2013-04-18T08:19:40Z")
Not really, only default ones. Not using area tables ether. ---
**Author:** ![harveyzh](https://avatars.discourse-cdn.com/v4/letter/h/258eb7/32.png) [@harveyzh](https://forums.percona.com/u/harveyzh)\ **Post date:** [April 19, 2013, 4:29am UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636/4 "2013-04-19T04:29:35Z")
I also have a problem. Packages info: Percona-Server-server-55-5.5.29-rel29.4.401.rhel6.x86\_64 percona-xtrabackup-2.0.5-499.rhel6.x86\_64 Commands: # cat /data/backup/mysql/2013-04-19\_11-04-34/xtrabackup\_checkpoints backup\_type = full-prepared from\_lsn = 0 to\_lsn = 308184144593 last\_lsn = 308882154381 # innobackupex --apply-log --redo-only --use-memory=16G /data/backup/mysql/2013-04-19\_11-04-34/ --incremental-dir=/data/backup/mysql/2013-04-19\_13-42-17 Partial error log: … … … InnoDB: Doing recovery: scanned up to log sequence number 309111833600 (88 %) InnoDB: Doing recovery: scanned up to log sequence number 309114524155 (88 %) 130419 17:33:34 InnoDB: Starting an apply batch of log records to the database… InnoDB: Progress in percents: 0 1 InnoDB: Probable data corruption on page 1662 InnoDB: Original record PHYSICAL RECORD: n\_fields 7; 1-byte offsets; info bits 32 0: len 4; hex 00000043; asc C;; 1: len 1; hex 00; asc ;; 2: len 4; hex 000047a0; asc G ;; 3: len 22; hex 00000001860300048000860800088000860800088000; asc ;; 4: len 4; hex 81757165; asc uqe;; 5: len 8; hex 8000013e1e903b68; asc \> ;h;; 6: len 8; hex 800000000042e94f; asc B O;; InnoDB: on that page. InnoDB: Cannot find the dir slot for record PHYSICAL RECORD: n\_fields 7; 1-byte offsets; info bits 0 0: len 4; hex 00000043; asc C;; 1: len 1; hex 00; asc ;; 2: len 4; hex 000047ce; asc G ;; 3: len 22; hex 00000001860300048000860800088000860800088000; asc ;; 4: len 4; hex 81828972; asc r;; 5: len 8; hex 8000013e1ea8e7d0; asc \> ;; 6: len 8; hex 800000000042e9e9; asc B ;; InnoDB: on that page! 130419 17:33:36 InnoDB: Page dump in ascii and hex (16384 bytes): … … … InnoDB: End of page dump 130419 17:33:36 InnoDB: Page checksum 2139845666 (32bit\_calc: 3413430305), prior-to-4.0.14-form checksum 705155133 InnoDB: stored checksum 700663176, prior-to-4.0.14-form stored checksum 71 InnoDB: Page lsn 71 3951049913, low 4 bytes of lsn at page end 3951049913 InnoDB: Page number (if stored to page already) 1662, InnoDB: space id (if created with \>= MySQL-4.1.1 and stored already) 0 InnoDB: Page may be an index page where index id is 18446744069414584320 InnoDB: (index “CLUST\_IND” of table “SYS\_IBUF\_TABLE”) 130419 17:33:36 InnoDB: Assertion failure in thread 140549583533824 in file page0page.c line 153 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to [URL=“[http://bugs.mysql.com](http://bugs.mysql.com)”][http://bugs.mysql.com](http://bugs.mysql.com)[/URL=“[http://bugs.mysql.com](http://bugs.mysql.com)”]. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: [URL=“[http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html"]http://dev.mysql.com/doc/refman/5.5/...-recovery.html[/URL="http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html](http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html%22%5Dhttp://dev.mysql.com/doc/refman/5.5/...-recovery.html%5B/URL=%22http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html)”] InnoDB: about forcing recovery. innobackupex: Error: innobackupex: ibbackup failed at /usr/bin/innobackupex line 381. ---
**Author:** ![jscheufler](https://avatars.discourse-cdn.com/v4/letter/j/5fc32e/32.png) [@jscheufler](https://forums.percona.com/u/jscheufler)\ **Post date:** [April 30, 2013, 4:22am UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636/5 "2013-04-30T04:22:18Z")
I do encounter the same problem, like DmitryK - however, I do encounter it on MySQL instead of mariaDB. First of all, my versions: MySQL 5.5.27 Percona XtraBackup 2.0.6 What’s strange in my case, is the fact that the backups for two of the three instances on my backup node can be restored without any hassle - but the third, and largest one ( about 1.2TB data-dir / 437GB zipped backup ) ultimately fails with the above mentioned assertion failure: ```auto 130426 9:04:02 InnoDB: Assertion failure in thread 139909354481408 in file xtrabackup.c line 1601 InnoDB: Failing assertion: cset == 0 ``` I am using the following statements to create and restore the backup Creation: ```auto innobackupex --slave-info --safe-slave-backup --stream=tar ./ | pigz -p 5 | split -d -b 100G - $FILENAME ``` Restore: ```auto cat 2013-04-23--11-51-59.xtrabackup.tar.gz* | pigz -dc | tar xif - -C /export/backup-tmp/ ``` Any advice would be appreciated. Greetings, Jan ---
**Author:** ![DmitryK](https://avatars.discourse-cdn.com/v4/letter/d/e47c2d/32.png) [@DmitryK](https://forums.percona.com/u/DmitryK)\ **Post date:** [April 30, 2013, 9:00am UTC](https://forums.percona.com/t/xtrabackup-is-failing-once-log-is-applied/2636/6 "2013-04-30T09:00:07Z")
I replaced MariaDB with Percona 5.5.30-30.2 and installed xtrabackup 2.0.6 - same problem with normal (full, no streaming) backup!