FIXED BUT NOT SOLVED
I got xtrabackup to run after setting it up properly - but ti died and apparently has left tables locked in all databases. (show open tables does not show all tables and access is denied to alo the affected databases)
Anyone know the best and safest way to unlock them ?? (phpmyadmin also became inaccessible through cPanel - and lands on the root login page insted)
Here is the run paste (actual username and password changed or xxxxxx’ed out)
****PASTE STARTS ****
root@main [~]# innobackupex --user=USERNAME --password=XXXXXXXXXXX* /data/backups
InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.
This software is published under
the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
161030 21:58:45 innobackupex: Starting mysql with options: --password=xxxxxxxx --user=‘USERNAME’ --unbuffered –
161030 21:58:45 innobackupex: Connected to database with mysql child process (pid=10344)
161030 21:58:51 innobackupex: Connection to database server closed
IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints “completed OK!”.
innobackupex: Using mysql Ver 15.1 Distrib 10.0.28-MariaDB, for Linux (x86_64) using readline 5.1
innobackupex: Using mysql server version Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
innobackupex: Created backup directory /data/backups/2016-10-30_21-58-51
161030 21:58:51 innobackupex: Starting mysql with options: --password=xxxxxxxx --user=‘bkupbigb’ --unbuffered –
161030 21:58:51 innobackupex: Connected to database with mysql child process (pid=10375)
161030 21:58:53 innobackupex: Connection to database server closed
161030 21:58:53 innobackupex: Starting ibbackup with command: xtrabackup_56 --defaults-group=“mysqld” --backup --suspend-at-end --target-dir=/data/backups/2016-10-30_21-58-51 --tmpdir=/tmp
innobackupex: Waiting for ibbackup (pid=10382) to suspend
innobackupex: Suspend file ‘/data/backups/2016-10-30_21-58-51/xtrabackup_suspended’
xtrabackup_56 version 2.0.8 for MySQL server 5.6.10 Linux (x86_64) (revision id: 587)
xtrabackup: uses posix_fadvise().
xtrabackup: cd to /var/lib/mysql
xtrabackup: Target instance is assumed as followings.
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 268435456
xtrabackup: using O_DIRECT
log scanned up to (1440614805348)
InnoDB: Allocated tablespace 137281, old maximum was 0
2016-10-30 21:58:53 7f3013e7e720 InnoDB: Operating system error number 24 in a file operation.
InnoDB: Error number 24 means ‘Too many open files’.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.6/…ror-codes.html
InnoDB: Error: could not open single-table tablespace file ./<DATABASE_NAME>/page_title.ibd
InnoDB: We do not continue the crash recovery, because the table may become
InnoDB: corrupt if we cannot apply the log records in the InnoDB log to it.
InnoDB: To fix the problem and start mysqld:
InnoDB: 1) If there is a permission problem in the file and mysqld cannot
InnoDB: open the file, you should modify the permissions.
InnoDB: 2) If the table is not needed, or you can restore it from a backup,
InnoDB: then you can remove the .ibd file, and InnoDB will do a normal
InnoDB: crash recovery and ignore that table.
InnoDB: 3) If the file system or the disk is broken, and you cannot remove
InnoDB: the .ibd file, you can set innodb_force_recovery > 0 in my.cnf
InnoDB: and force InnoDB to continue crash recovery here.
innobackupex: Error: ibbackup child process has died at /usr/bin/innobackupex line 386.
root@main [~]#
****PASTE STOPS ****
***** EDIT - EDIT - EDIT ****
forcing a cPanel update apparently created an initiation of a transaction and cleared the locks on all affected databases and fixed phpmyadmin access - so it fixed the locked up issue and th websites came back up but it still never solved why the xtrabackup died in the first place