Seems that XtraDB can’t get the list of IPs using ifconfig:
130320 18:49:37 [ERROR] WSREP: Process completed with error: /sbin/ifconfig | grep -E ‘[1]+inet addr:’ | grep -m1 -v ‘inet addr:127’ | sed ‘s/ /’ | awk ‘{ print $3 }’: 2 (No such file or directory)
it should be able to run the tool to get the information it needs. You can also use wsrep_node_address to set the IP in which rsync and other daemons should listen.
I have the same problem (using rsync or xtrabackup sst methods).
I used wsrep_node_address to mitigate the first error but can’t get around the second: 130320 18:49:38 [Warning] WSREP: Gap in state sequence. Need state transfer. 130320 18:49:40 [Note] WSREP: Running: ‘wsrep_sst_rsync --role ‘joiner’ --address ‘192.168.133.70:4567’ --auth ‘’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/my.cnf’ --parent ‘38994’’ 130320 18:49:40 [ERROR] WSREP: Failed to read 'ready ’ from: wsrep_sst_rsync --role ‘joiner’ --address ‘192.168.133.70:4567’ --auth ‘’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/my.cnf’ --parent ‘38994’
Read: ‘(null)’
130320 18:49:40 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role ‘joiner’ --address ‘192.168.133.70:4567’ --auth ‘’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/my.cnf’ --parent ‘38994’: 2 (No such file or directory)
130320 18:49:40 [ERROR] WSREP: Failed to prepare for ‘rsync’ SST. Unrecoverable.
130320 18:49:40 [ERROR] Aborting
Did you have any luck? I’m guessing the mysql process doesn’t have permissions to run/access both /bin/ip and wsrep_sst_rsync but I haven’t yet found a way to solve it (chown’d and chmod’d everything I can find that could be causing it).
I have 3 nodes, they were all working beautifully until a switch died taking out all but one node.
I made that node primary with gcomm:// and have since been trying to add the other two back to the cluster but I keep having problems.
Firstly I was getting the one Miguel identified above and I mitigated that by using wsrep_node_address, but now I can’t get past ‘No such file or directory’ as demonstrated below:
Using xtrabackup: [INDENT]130530 15:31:50 [Warning] WSREP: Gap in state sequence. Need state transfer.
130530 15:31:50 [Note] WSREP: Setting wsrep_ready to 0
130530 15:31:50 [Note] WSREP: [debug]: closing client connections for PRIM
130530 15:31:52 [Note] WSREP: waiting for client connections to close: 2
130530 15:31:52 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role ‘joiner’ --address ‘176.123.62.2’ --auth ‘repl:’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘24455’’
130530 15:31:52 [ERROR] WSREP: Failed to read ‘ready ’ from: wsrep_sst_xtrabackup --role ‘joiner’ --address ‘176.123.62.2’ --auth 'repl:’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘24455’
Read: ‘(null)’
130530 15:31:52 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role ‘joiner’ --address ‘176.123.62.2’ --auth ‘repl:****’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘24455’: 2 (No such file or directory)
130530 15:31:52 [ERROR] WSREP: Failed to prepare for ‘xtrabackup’ SST. Unrecoverable.
130530 15:31:52 [ERROR] Aborting[/INDENT]
Using rsync:
[INDENT]130530 14:28:46 [Note] WSREP: State transfer required:
Group state: 079517f3-9dd3-11e2-0800-91e2f9f7eca8:42281
Local state: 079517f3-9dd3-11e2-0800-91e2f9f7eca8:19746
130530 14:28:46 [Note] WSREP: New cluster view: global state: 079517f3-9dd3-11e2-0800-91e2f9f7eca8:42281, view# 10: Primary, number of nodes: 2, my index: 1, protocol version 2
130530 14:28:46 [Warning] WSREP: Gap in state sequence. Need state transfer.
130530 14:28:48 [Note] WSREP: Running: ‘wsrep_sst_rsync --role ‘joiner’ --address ‘176.123.62.2’ --auth ‘repl:’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘19421’’
130530 14:28:48 [ERROR] WSREP: Failed to read ‘ready ’ from: wsrep_sst_rsync --role ‘joiner’ --address ‘176.123.62.2’ --auth 'repl:’’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘19421’
Read: ‘(null)’
130530 14:28:48 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role ‘joiner’ --address ‘176.123.62.2’ --auth ‘repl:’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘19421’: 2 (No such file or directory)
130530 14:28:48 [ERROR] WSREP: Failed to prepare for ‘rsync’ SST. Unrecoverable.
130530 14:28:48 [ERROR] Aborting
[/INDENT]
In my quest to find the problem I’ve run the above command at the shell
[INDENT]# wsrep_sst_rsync --role ‘joiner’ --address ‘176.123.62.2’ --auth 'repl:’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/mysql/my.cnf’ --parent ‘20314’
ready 176.123.62.2:4444/rsync_sst
WSREP_SST: [ERROR] Parent mysqld process (PID:20314) terminated unexpectedly. (20130530 16:44:21.212)
WSREP_SST: [INFO] Joiner cleanup. (20130530 16:44:21.214)
done.
[/INDENT]
As you can see it works as expected and returns the WSREP seems to be expecting (other than a complaint that the PID of mysql isn’t there - because it isn’t as mysql has aborted).
This is happening from two different nodes with the same error (both Ubuntu 12.04). I’ve upgraded one to 2.1.3 and the other two are still running 2.0.6.
To me it looks like there’s a problem spawning wsrep_sst_rsync within mysql but I’m having no luck finding the cause… I’ve also tried symlinking wsrep_sst* and innobackup* to /usr/sbin and mysql’s data dir to no avail.
I expect there’s something stupid that I’m missing but I just can’t find it,
/’ | awk ‘{ print $3 }’: 2 (No such file or directory)