State Transfer Problem

Percona XtraDB Cluster from the Percona repo (percona-xtradb-cluster-server-5.5/5.5.27-23.6-356.oneiric) on Ubuntu 11.10 64-bit.

Error is “[ERROR] WSREP: async IST sender failed to serve tcp://x.x.x.x:4568: ist send failed: 1’, asio error ‘Connection reset by peer’: 104 (Connection reset by peer) at galera/src/ist.cpp:send():740”. Node fails to fall back to full state transfer and mysql service fails to start. No innobackup or sst error logs are created after failure.


When the original problem happens again, can you post the contents of the innobackup.backup.log from the donor. This is found within the data directory.


From this log on the donor I see the below, so it may have the log file available. Let us know what you find out :slight_smile:

130513 19:01:41 [ERROR] WSREP: Failed to read from: wsrep_sst_xtrabackup 'donor' '' '(null)' '/var/lib/mysql/' '/etc/mysql/my.cnf' '3214b7d4-8b59-11e2-0800-fde78b56d029' '109567765' '1'
130513 19:01:41 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup 'donor' '' '(null)' '/var/lib/mysql/' '/etc/mysql/my.cnf' '3214b7d4-8b59-11e2-0800-fde78b56d029' '109567765' '1': 1 (Operation not permitted)
130513 19:01:41 [Warning] WSREP: 0 (db-2): State transfer to 1 (db-3) failed: -1 (Operation not permitted)

After upgrade to 5.5.30-rel30.2 initial start/joining the cluster state transfer failed with “InnoDB: Error: io_setup() failed with EAGAIN after 5 attempts.” More information is available at this URL:…in-particular/. Increased the fs.aio-max-nr in sysctl.conf and the transfer worked on the next try.

Just wanted to say that I have upgraded all of my nodes/clusters to 5.5.30-rel30.2 I have not had a single problem since so there is clearly something wrong with the old version. If anyone is still running on 5.5.27 you should probably upgrade as soon as you can.