IST first seqno 7 not found from cache, falling back to SST

xtrabackup: version 2.0.2
MySQL: version 5.5.27
Distro: CentOS 5.7

my.cnf on the donor:

[mysqld]binlog_format=ROWwsrep_provider=/usr/lib64/libgalera_smm.sowsrep_provider_options="gcache.size=4G;"wsrep_slave_threads=2wsrep_cluster_name=trimethylxanthinewsrep_sst_method=xtrabackupwsrep_sst_auth=root:xwsrep_node_name=node1wsrep_node_address=192.168.5.148innodb_locks_unsafe_for_binlog=1innodb_autoinc_lock_mode=2[mysqld_safe]wsrep_urls=gcomm://192.168.5.148:4567,gcomm://192.168.5.149:4567,gcomm://192.168.5.197:4567,gcomm://

my.cnf on the joiner:

[mysqld]binlog_format=ROWwsrep_provider=/usr/lib64/libgalera_smm.sowsrep_slave_threads=2wsrep_cluster_name=trimethylxanthinewsrep_sst_method=xtrabackupwsrep_sst_auth=root:xwsrep_node_name=node2wsrep_node_address=192.168.5.197innodb_locks_unsafe_for_binlog=1innodb_autoinc_lock_mode=2[mysqld_safe]wsrep_urls=gcomm://192.168.5.148:4567,gcomm://192.168.5.149:4567,gcomm://192.168.5.197:4567,gcomm://

The backup are performed by running:

innobackupex --galera-info --stream=tar ./ | gzip - | nc 192.168.5.197 9999

grastate.dat on the joiner (sure, take it from the xtrabackup_galera_info)

GALERA saved stateversion: 2.1uuid: f68ad1a2-0a4e-11e2-0800-7441f90e56cfseqno: 6cert_index:

mysqld.log on the donor:

120930 17:00:49 [Note] WSREP: Shifting SYNCED → DONOR/DESYNCED (TO: 8)120930 17:00:49 [Note] WSREP: IST request: f68ad1a2-0a4e-11e2-0800-7441f90e56cf:6-8|tcp://192.168.5.197:4568120930 17:00:49 [Note] WSREP: IST first seqno 7 not found from cache, falling back to SST

mysqld.log on the joiner:

120930 17:01:54 [Note] WSREP: Prepared SST request: xtrabackup|192.168.5.197:4444/xtrabackup_sst120930 17:01:54 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.120930 17:01:54 [Note] WSREP: Assign initial position for certification: 8, protocol version: 2120930 17:01:54 [Note] WSREP: Prepared IST receiver, listening at: tcp://192.168.5.197:4568120930 17:01:54 [Note] WSREP: Node 2 (SVR282-5197.localdomain) requested state transfer from ‘any’. Selected 0 (x)(SYNCED) as donor.120930 17:01:54 [Note] WSREP: Shifting PRIMARY → JOINER (TO: 8)120930 17:01:54 [Note] WSREP: Requesting state transfer: success, donor: 0120930 17:03:18 [Note] WSREP: 0 (SVR120-27148.localdomain): State transfer to 2 (SVR282-5197.localdomain) complete.120930 17:03:18 [Note] WSREP: Member 0 (SVR120-27148.localdomain) synced with group.120930 17:03:29 [Note] WSREP: SST complete, seqno: 8

The second node joined the cluster successfully but it does a SST instead of an IST. Why it doesn’t find the sequence number from cache? My database is only ~ 3GB, there is a little changes from the time of the backup until the restore is complete.

sst.err on the joiner is empty.
Some lines at the end of innobackup.backup.log on the donor:

…innobackupex: Resuming ibbackup xtrabackup: The latest check point (for incremental): '1850952486’xtrabackup: Stopping log copying thread…>> log scanned up to (1850952640) xtrabackup: Streaming transaction log from a temporary file…xtrabackup: Done.xtrabackup: Transaction log of lsn (1850952486) to (1850952640) was copied.120930 18:52:04 innobackupex: All tables unlocked120930 18:52:04 innobackupex: Connection to database server closed innobackupex: Backup created in directory '/tmp’innobackupex: MySQL binlog position: filename ‘mysql-bin.000185’, position 107innobackupex: You must use -i (–ignore-zeros) option for extraction of the tar stream.120930 18:52:04 innobackupex: completed OK!