WSREP: Failed to read uuid:seqno from joiner script

MySQL & MariaDB Percona XtraDB Cluster 5.x
1

hey guys,

so i have a 2 nodes cluster. i rebooted each of the nodes yesterday and node 2 doesn’t want to rejoin the cluster. the my.cnf are both the same auth wise, and everything has been working perfectly till the reboot.

there was a problem with node2 tho, it ran a fsck and the timezone got desynced. so i fixed that, and when trying to rejoin the node to the cluster i get:

2015-01-30 17:58:13 19301 [Note] WSREP: Member 0.0 (streamer25) synced with group.
WSREP_SST: [ERROR] xtrabackup process ended without creating ‘/var/lib/mysql//xtrabackup_galera_info’ (20150130 17:58:13.032)
WSREP_SST: [INFO] Contents of datadir (20150130 17:58:13.034)
WSREP_SST: [INFO] -rw-rw---- 1 mysql mysql 8636 Dec 24 08:00 /var/lib/mysql//adreport/admin.frm

2015-01-30 17:58:13 19301 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role ‘joiner’ --address ‘192.168.1.26’ --auth ‘replica:PEYk7afduwE4RB’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/my.cnf’ --parent ‘19301’ ‘’ : 32 (Broken pipe)
2015-01-30 17:58:13 19301 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2015-01-30 17:58:13 19301 [ERROR] WSREP: SST failed: 32 (Broken pipe)
2015-01-30 17:58:13 19301 [ERROR] Aborting

now. i tried to get an innobackupex full backup from node1 and load it on node2 but i still get the same error.

can anyone point me into what i might be doing wrong please?

2

just a quick update so: if i take the innobackupex full backup from node 1 and try to bootstrap node2 it works perfectly fine, however if i try to restart node2 sql i get:

2015-01-30 18:37:32 7f9716a837e0 InnoDB: Error: page 7 log sequence number 550583039327
InnoDB: is in the future! Current system log sequence number 549779227616.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: [url]http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html[/url]
InnoDB: for more information.
2015-01-30 18:37:32 7f9716a837e0 InnoDB: Error: page 2 log sequence number 550678625752
InnoDB: is in the future! Current system log sequence number 549779227616.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: [url]http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html[/url]
InnoDB: for more information.
2015-01-30 18:37:32 7f9716a837e0 InnoDB: Error: page 4 log sequence number 550607253129
InnoDB: is in the future! Current system log sequence number 549779227616.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: [url]http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html[/url]
InnoDB: for more information.
2015-01-30 18:37:32 7f9716a837e0 InnoDB: Error: page 11 log sequence number 550583154624
InnoDB: is in the future! Current system log sequence number 549779227616.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: [url]http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html[/url]
InnoDB: for more information.

but both servers are in sync time so i just don’t get it.

3

i didn’t find any solution to this so what i did was … i loaded 2 virtual machines, made a cluster, then took a innobackupex from production and loaded it there, and joined them both just because i thought there might be something wrong with the innobackupex backup. everything went fine so this evening when traffic was low, i took an innobackupex from node1, loaded it on node2 in bootstrap, then renamed node1 mysql folder, took another innobackupex from node2, loaded it on node1 and then once the sync was complete i restarted node2 normally.

i also changed the xtrabackup method from xtrabackup to xtrabackup-v2. not sure if it had any impact but this is how i sorted the issue. didn’t find anything online that was really relevant to what i have been experiencing here.

if we could get any insight on what could have been wrong, that would be great.

4

I’ve had the same issue and this was the fix. wsrep_sst_method=xtrabackup-v2