For smaller DBs I don’t see this, but while working with an 8TB MariaDB, I am running into a infinite loop when running a FULL backup. The backup seems to almost complete, where the backup directory has the 8TB of data backup created, but continue to have a log file scanned.
$ innobackupex -v
innobackupex version 2.4.3 Linux (x86_64) (revision id: 6a46905)
mysql@hlxtil0694 /opt/app/mysql/local/bin
$
${MySQL_Backup_Bin}/innobackupex --defaults-file=${MYSQL_CONF_FILE} --user=${MYSQL_USER} --password=${MySQL_PWD} --history=${HOSTNAME}-${MySQL_BACKUP} --parallel=10 ${MySQL_FULL_BackupDir} >> ${TMP_LOG} 2>&1
<><><><> [Wed Sep 28 04:10:02 PDT 2016] Starting FULL Backup
Flushing Logs…
Flush Logs Done
Beginning Backup
Running with Parallel=10…
160928 04:10:02 innobackupex: Starting the backup operation
IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints “completed OK!”.
160928 04:10:02 version_check Connecting to MySQL server with DSN ‘dbi:mysql:;mysql_read_default_group=xtrabackup;port=3306;mys
ql_socket=/opt/app/mysql/mysql.sock’ as ‘root’ (using password: YES).
160928 04:10:02 version_check Connected to MySQL server
160928 04:10:02 version_check Executing a version check against the server…
160928 04:10:02 version_check Done.
160928 04:10:02 Connecting to MySQL server host: localhost, user: root, password: set, port: 3306, socket: /opt/app/mysql/mysql.
sock
Using server version 10.1.14-MariaDB-enterprise
/opt/app/mysql/product/xtrabackup-2.4.3/bin/innobackupex version 2.4.3 based on MySQL server 5.7.11 Linux (x86_64) (revision id:
6a46905)
xtrabackup: uses posix_fadvise().
xtrabackup: cd to /opt/app/mysql/data
xtrabackup: open files limit requested 8000, set to 65536
xtrabackup: using the following InnoDB configuration:
xtrabackup: innodb_data_home_dir = /opt/app/mysql/data
xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend:max:16G
xtrabackup: innodb_log_group_home_dir = /opt/app/mysql/iblogs
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 1073741824
InnoDB: Number of pools: 1
160928 04:10:02 >> log scanned up to (14766689085698)
…
(last backup file written to backup dir at 08:00)
…
60928 21:35:48 >> log scanned up to (14766689085698)
160928 21:35:49 >> log scanned up to (14766689085698)
160928 21:35:50 >> log scanned up to (14766689085698)
160928 21:35:51 >> log scanned up to (14766689085698)
160928 21:35:52 >> log scanned up to (14766689085698)
160928 21:35:53 >> log scanned up to (14766689085698)
160928 21:35:54 >> log scanned up to (14766689085698)
160928 21:35:55 >> log scanned up to (14766689085698)
160928 21:35:56 >> log scanned up to (14766689085698)
(and this entry will just continue)
I found a bug similar to this from a few years ago: https://bugs.launchpad.net/percona-xtrabackup/+bug/1007446.
I searched for the xtrabackup_suspended file, but could not find it. This issue seems to appear randomly. I have hit this issue 3 of the last 4 backups executed.
Any ideas?
Thanks in advance, Jim