Using innobackupex-1.5.1 failing with this report.

What could be causing this errors ?. I was able to successfully restore a backup before o a single mysql instance, but we need this to work on mysqld_multi environment.

130914 11:53:43 innobackupex-1.5.1: Starting ibbackup with command: xtrabackup --defaults-file=“/etc/my.cnf” --defaults-group=“mysqld2” --prepare --target-dir=/home/backup/backupssave/PT/xtrabackupsincrementalwork/xtrabackupsincremental/full/2013-09-13_07-55-30 --apply-log-only --use-memory=512M --tmpdir=/tmp

xtrabackup version 2.1.4 for Percona Server 5.1.70 unknown-linux-gnu (x86_64) (revision id: 656)
xtrabackup: cd to /home/backup/backupssave/PT/xtrabackupsincrementalwork/xtrabackupsincremental/full/2013-09-13_07-55-30
xtrabackup: This target seems to be not prepared yet.
xtrabackup: xtrabackup_logfile detected: size=67305472, start_lsn=(1732818865612)
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 67305472
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 67305472
xtrabackup: Starting InnoDB instance for recovery.
xtrabackup: Using 536870912 bytes for buffer pool (set by --use-memory parameter)
InnoDB: The InnoDB memory heap is disabled
InnoDB: Mutexes and rw_locks use GCC atomic builtins
InnoDB: Compressed tables use zlib 1.2.3
130914 11:53:43 InnoDB: Initializing buffer pool, size = 512.0M
130914 11:53:43 InnoDB: Completed initialization of buffer pool
130914 11:53:43 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 1732818865612
130914 11:53:43 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files…
InnoDB: Doing recovery: scanned up to log sequence number 1732824108032 (8 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732829350912 (17 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732834593792 (26 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732839836672 (35 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732845079552 (43 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732850322432 (52 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732855565312 (61 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732860808192 (70 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732866051072 (78 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732871293952 (87 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732876536832 (96 %)
InnoDB: Doing recovery: scanned up to log sequence number 1732878690523 (99 %)
130914 11:53:51 InnoDB: Starting an apply batch of log records to the database…
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed

[notice (again)]
If you use binary log and don’t use any hack of group commit,
the binary log position seems to be:

xtrabackup: starting shutdown with innodb_fast_shutdown = 1
130914 11:54:14 InnoDB: Starting shutdown…
130914 11:54:14 InnoDB: Assertion failure in thread 140483548915456 in file ut/ut0mem.c line 103
InnoDB: Failing assertion: ret || !assert_on_error
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: [url]http://dev.mysql.com/doc/refman/5.1/en/forcing-innodb-recovery.html[/url]
InnoDB: about forcing recovery.
innobackupex-1.5.1: Error:
innobackupex-1.5.1: ibbackup failed at /usr/bin/innobackupex-1.5.1 line 416.

Hi,

As per this error msg “InnoDB: Assertion failure in thread 140483548915456 in file ut/ut0mem.c line 103”, seems probably it’s related to insufficient memory. Can you check memory consumption when you are running this on server next time? (i.e free -m). As you are running mysql_multi, there will be more than one mysql instances on server which can consume more memory of that server.