XtraDB Cluster node disconnect from cluster after SST and need new SST operation

XtraDB Cluster node db1 try to connect to cluster (db3, garbd), but after success SST and established connection (wsrep_cluster_size show 3 nodes), db1 disconnect from cluster and need another SST (Remove ‘/home/database/mysql//grastate.dat’ file and restart if you wish to continue. (FATAL)).

Cluster worked fine for 2 months.

Following important options are set (DB1):
wsrep_sst_method=xtrabackup
wsrep_cluster_address=gcomm://192.168.1.85,192.168.1.87,192.168.1.60
wsrep_sst_donor=db3_node
wsrep_causal_reads=1
wsrep_provider_options=“gcache.size=4G”

Warning and error messages.

DB1:

DB3:

Generally when Galera tells you things like this, you should listen.

I also noted these messages:

130806 10:27:37 [Warning] Could not increase number of max_open_files to more than 65535 (request: 1049087)

That makes me suspect you have an open files limit problem on DB1 – have you adjusted your ulimits?