[HELP] Error: wsrep_sst_xtrabackup --role 'donor' ~~~2 (No such file or directory)]

[clh.joe]

I just meet error:
[Process completed with error: wsrep_sst_xtrabackup --role ‘donor’ --address ‘10.109.37.4:4444/xtrabackup_sst’ --auth ‘root:xxx’ --socket ‘/var/run/mysqld/mysqld.sock’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --gtid ‘edefb40a-bc62-11e2-0800-9c175fdcc5a1:14’: 2 (No such file or directory)]

could somebody please help me~~~

below is my configuration and logs. Thank you !

OS: Ubuntu 12.10 x86_64
Percona version: 5.5.30 [DEB]
my.cnf permission:root:root 644
Disable appamor(# /etc/init.d/appamor stop)

node1(10.109.37.3):

[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
wsrep_provider=/usr/lib/libgalera_smm.so
wsrep_slave_threads=8
wsrep_cluster_address=gcomm://
wsrep_sst_method=xtrabackup
wsrep_cluster_name = “Percona-cluster”
binlog_format=ROW
default_storage_engine=InnoDB
innodb_locks_unsafe_for_binlog=1
innodb_autoinc_lock_mode=2
wsrep_sst_auth=root:xxx

bind-address = 0.0.0.0
!includedir /etc/mysql/conf.d/

[B]node2(10.109.37.4):[/B]

[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

wsrep_provider=/usr/lib/libgalera_smm.so
wsrep_slave_threads=8
wsrep_sst_method=xtrabackup
wsrep_cluster_address=gcomm://10.109.37.3
binlog_format=ROW
default_storage_engine=InnoDB
innodb_locks_unsafe_for_binlog=1
wsrep_cluster_name = "Percona-cluster"
innodb_autoinc_lock_mode=2
wsrep_sst_auth=root:xxx
 


bind-address = 0.0.0.0
!includedir /etc/mysql/conf.d/

[B]node1/B
/var/log/mysql/error.log

130514 18:35:38 [Note] WSREP: Running: ‘wsrep_sst_xtrabackup --role ‘donor’ --address ‘10.109.37.4:4444/xtrabackup_sst’ --auth ‘root:xxx’ --socket ‘/var/run/mysqld/mysqld.sock’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --gtid ‘edefb40a-bc62-11e2-0800-9c175fdcc5a1:14’’

130514 18:35:38 [ERROR] WSREP: Failed to read from: wsrep_sst_xtrabackup --role 'donor' --address '10.109.37.4:4444/xtrabackup_sst' --auth 'root:xxx' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid 'edefb40a-bc62-11e2-0800-9c175fdcc5a1:14'


130514 18:35:38 [Note] WSREP: sst_donor_thread signaled with 0


130514 18:35:38 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'donor' --address '10.109.37.4:4444/xtrabackup_sst' --auth 'root:xxx' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid 'edefb40a-bc62-11e2-0800-9c175fdcc5a1:14': 2 (No such file or directory)


130514 18:35:38 [Warning] WSREP: 1 (openstack-slave2): State transfer to 0 (openstack-slave2) failed: -1 (Operation not permitted)

[B]node2[/B](10.109.37.4)
/var/log/mysql/error.log

	130514 18:35:39 [Note] WSREP: New cluster view: global state: edefb40a-bc62-11e2-0800-9c175fdcc5a1:14, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 2


	130514 18:35:39 [Warning] WSREP: Gap in state sequence. Need state transfer.


	130514 18:35:41 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '10.109.37.4' --auth 'root:xxx' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '2464''


	130514 18:35:41 [Note] WSREP: Prepared SST request: xtrabackup|10.109.37.4:4444/xtrabackup_sst


	130514 18:35:41 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.


	130514 18:35:41 [Note] WSREP: Assign initial position for certification: 14, protocol version: 2


	130514 18:35:41 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (edefb40a-bc62-11e2-0800-9c175fdcc5a1): 1 (Operation not permitted)


	at galera/src/replicator_str.cpp:prepare_for_IST():436. IST will be unavailable.


	130514 18:35:41 [Note] WSREP: Node 0 (openstack-slave2) requested state transfer from '*any*'. Selected 1 (openstack-slave2)(SYNCED) as donor.


	130514 18:35:41 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 14)


	130514 18:35:41 [Note] WSREP: Requesting state transfer: success, donor: 1


	130514 18:35:41 [Warning] WSREP: 1 (openstack-slave2): State transfer to 0 (openstack-slave2) failed: -1 (Operation not permitted)


	130514 18:35:41 [ERROR] WSREP: gcs/src/gcs_group.c:gcs_group_handle_join_msg():719: Will never receive state. Need to abort.

[revin]

Can you post the sst.err and innobackup.backup.log from the donor (node1 inside /var/lib/mysql)

[clh.joe]

I can’t find sst.err on my server@@. I even tried # find / -name “sst.err” and found nothing. Below is my innobackup.backup.log from the donor. Thank you!!

	InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy


	and Percona Ireland Ltd 2009-2012. All Rights Reserved.


	 


	This software is published under


	the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.


	 


	130514 18:34:05 innobackupex: Starting mysql with options: --defaults-file='/etc/mysql/my.cnf' --password=xxxxxxxx --user='root' --socket='/var/run/mysqld/mysqld.sock' --unbuffered --


	130514 18:34:05 innobackupex: Connected to database with mysql child process (pid=26137)
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[revin]

@clh.joe;10140

I can’t find sst.err on my server@@. I even tried # find / -name “sst.err” and found nothing. Below is my innobackup.backup.log from the donor. Thank you!! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy and Percona Ireland Ltd 2009-2012. All Rights Reserved. This software is published under the GNU GENERAL PUBLIC LICENSE Version 2, June 1991. 130514 18:34:05 innobackupex: Starting mysql with options: --defaults-file=‘/etc/mysql/my.cnf’ --password=xxxxxxxx --user=‘root’ --socket=‘/var/run/mysqld/mysqld.sock’ --unbuffered – 130514 18:34:05 innobackupex: Connected to database with mysql child process (pid=26137) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hhmm, that’s odd, the log is truncated - can you try to run a manual xtrabackup backup on the donor?

[Scarlett]

No, you cannot. Despite the fact that each node has the same data, it is also true that each node has it’s own independent internal InnoDB structures.