# Cluster nodes keep crashing

**URL:** <https://forums.percona.com/t/cluster-nodes-keep-crashing/3934>\
**Category:** Percona XtraDB Cluster 5.x\
**Created:** [December 7, 2014, 3:13am UTC](https://forums.percona.com/t/cluster-nodes-keep-crashing/3934 "2014-12-07T03:13:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![r00f](https://avatars.discourse-cdn.com/v4/letter/r/74df32/32.png) [@r00f](https://forums.percona.com/u/r00f)\
**Post date:** [December 7, 2014, 3:13am UTC](https://forums.percona.com/t/cluster-nodes-keep-crashing/3934/1 "2014-12-07T03:13:49Z")

</div>

I have 3-node cluster. At some point it went bad, so after that it wast started again, with one node being started with ‘bootstrap-pxc’ option, and two others just syncing to it.  
Now I have two nodes crashing all the time, while the node which was started first is always ok.

On the crashing nodes, in the log I see the following:

2014-12-04 14:58:05 16834 [Warning] WSREP: BF applier failed to open\_and\_lock\_tables: 1615, fatal: 0 wsrep = (exec\_mode: 1 conflict\_state: 5 seqno: 204996287)  
2014-12-04 14:58:05 16834 [Warning] WSREP: RBR event 3 Delete\_rows apply warning: 1615, 204996287  
2014-12-04 14:58:05 16834 [Warning] WSREP: failed to replay trx: source: 8c76e5c3-7b99-11e4-b260-4be15515cbc1 version: 3 local: 1 state: REPLAYING flags: 1 conn\_id: 43055 trx\_id: 1783377670 seqnos (l: 80620, g: 204996287, s: 204996283, d: 204996171, ts: 21382320446607328)  
2014-12-04 14:58:05 16834 [Warning] WSREP: Failed to apply trx 204996287 4 times  
2014-12-04 14:58:05 16834 [ERROR] WSREP: trx\_replay failed for: 6, query: void  
2014-12-04 14:58:05 16834 [ERROR] Aborting

This error almost always shows ‘Delete\_rows apply warning’, however the general-log show no delete queries at all.  
The average life of crashing node is about 15-20 minutes, however when I run it inside the gdb trying to catch the problem, it works for days without any problems, which makes the situation even more strange.