# PXC 5.6.24 - BF applier failed to open\_and\_lock\_tables

**URL:** https://forums.percona.com/t/pxc-5-6-24-bf-applier-failed-to-open-and-lock-tables/4295
**Category:** Percona XtraDB Cluster 5.x
**Created:** [July 9, 2015, 11:59am UTC](https://forums.percona.com/t/pxc-5-6-24-bf-applier-failed-to-open-and-lock-tables/4295 "2015-07-09T11:59:41Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![Xengulai](https://avatars.discourse-cdn.com/v4/letter/x/9f8e36/32.png) [@Xengulai](https://forums.percona.com/u/Xengulai)
#### Post date: [July 9, 2015, 11:59am UTC](https://forums.percona.com/t/pxc-5-6-24-bf-applier-failed-to-open-and-lock-tables/4295/1 "2015-07-09T11:59:41Z")

</div>

I am running a 3 node cluster of PXC and I keep getting random crashes on all 3 nodes.

Setup:  
Ubuntu 14.04.2 LTS

- 60 GB RAM
- SSD RAID 10 (130GB)  
PXC 5.6.24-72.2-56-log - Percona XtraDB Cluster (GPL), Release rel72.2, Revision 43abf03, WSREP version 25.11, wsrep\_25.11

> >

[mysqld]

# GENERAL

bind-address = 0.0.0.0  
character-set-server = utf8  
collation-server = utf8\_general\_ci  
default\_storage\_engine = InnoDB  
event-scheduler = ON  
pid-file = /var/run/mysqld/mysqld.pid  
port = 3306  
server-id = 1  
socket = /var/run/mysqld/mysqld.sock  
user = mysql

# MyISAM

key-buffer-size = 32M  
myisam-recover-options = FORCE,BACKUP

# SAFETY

innodb = FORCE  
innodb-strict-mode = 1  
max-allowed-packet = 64M  
max-connect-errors = 1000000  
skip-external-locking  
skip-host-cache  
skip-name-resolve  
sql-mode = STRICT\_TRANS\_TABLES,NO\_AUTO\_CREATE\_USER,NO\_AUTO\_VA LUE\_ON\_ZERO,NO\_ENGINE\_SUBSTITUTION  
sysdate-is-now = 1

# DATA STORAGE

datadir = /var/lib/mysql

# BINARY LOGGING

expire-logs-days = 14  
log-bin = /var/lib/mysql/mysql-bin  
log-slave-updates  
sync-binlog = 1

# CACHES AND LIMITS

back-log = 1000  
connect-timeout = 20  
interactive-timeout = 30  
join-buffer-size = 8M  
max-binlog-size = 100M  
max-connections = 2000  
max-heap-table-size = 32M  
open-files-limit = 65535  
preload-buffer-size = 65536  
query-cache-size = 0  
query-cache-type = 0  
sort-buffer-size = 2M  
read-buffer-size = 4M  
read-rnd-buffer-size = 4M  
table-definition-cache = 4096  
table-open-cache = 5000  
thread-cache-size = 100  
thread-stack = 256K  
tmp-table-size = 32M  
wait-timeout = 30

# INNODB

innodb-buffer-pool-instances = 8  
innodb-buffer-pool-size = 40G  
innodb-file-per-table = 1  
innodb-flush-log-at-trx-commit = 1  
innodb-flush-method = O\_DIRECT  
innodb-lock-wait-timeout = 15  
innodb-log-files-in-group = 2  
innodb-log-file-size = 512M

# LOGGING \*

log-error = /var/log/mysql/mysql-error.log  
log-queries-not-using-indexes = 0  
slow-query-log = 0

# WSREP

wsrep\_provider = /usr/lib/galera3/libgalera\_smm.so  
wsrep\_cluster\_address = gcomm://,,  
binlog\_format = ROW  
innodb\_autoinc\_lock\_mode = 2  
wsrep\_node\_address =   
wsrep\_node\_name = “db01”  
wsrep\_sst\_method = xtrabackup-v2  
wsrep\_cluster\_name =   
wsrep\_sst\_auth = “”  
wsrep\_slave\_threads = 8  
wsrep\_notify\_cmd = /etc/mysql/wsrep\_notify  
\<\<

I was getting crashes every 1-3 days on all 3 nodes until the release of PXC 5.6.24. Now I get them about once a week. At first, I thought it was a specific cron job because the crash timestamps had similar minutes, so I looked at the job but I couldn’t replicate the error manually. Then the crash timestamps started to be different so I haven’t been able to track any pattern. When the nodes crash, I get the same error across the board:

2015-07-09 00:44:36 19638 [Warning] WSREP: BF applier failed to open\_and\_lock\_tables: 1615, fatal: 0 wsrep = (exec\_mode: 1 conflict\_state: 5 seqno: 46677454)  
2015-07-09 00:44:36 19638 [Warning] WSREP: RBR event 3 Write\_rows apply warning: 1615, 46677454  
2015-07-09 00:44:36 19638 [Warning] WSREP: Failed to apply app buffer: seqno: 46677454, status: 1  
at galera/src/trx\_handle.cpp:apply():351  
Retrying 2th time  
2015-07-09 00:44:36 19638 [Warning] WSREP: BF applier failed to open\_and\_lock\_tables: 1615, fatal: 0 wsrep = (exec\_mode: 1 conflict\_state: 5 seqno: 46677454)  
2015-07-09 00:44:36 19638 [Warning] WSREP: RBR event 3 Write\_rows apply warning: 1615, 46677454  
2015-07-09 00:44:36 19638 [Warning] WSREP: Failed to apply app buffer: seqno: 46677454, status: 1  
at galera/src/trx\_handle.cpp:apply():351  
Retrying 3th time  
2015-07-09 00:44:36 19638 [Warning] WSREP: BF applier failed to open\_and\_lock\_tables: 1615, fatal: 0 wsrep = (exec\_mode: 1 conflict\_state: 5 seqno: 46677454)  
2015-07-09 00:44:36 19638 [Warning] WSREP: RBR event 3 Write\_rows apply warning: 1615, 46677454  
2015-07-09 00:44:36 19638 [Warning] WSREP: Failed to apply app buffer: seqno: 46677454, status: 1  
at galera/src/trx\_handle.cpp:apply():351  
Retrying 4th time  
2015-07-09 00:44:36 19638 [Warning] WSREP: BF applier failed to open\_and\_lock\_tables: 1615, fatal: 0 wsrep = (exec\_mode: 1 conflict\_state: 5 seqno: 46677454)  
2015-07-09 00:44:36 19638 [Warning] WSREP: RBR event 3 Write\_rows apply warning: 1615, 46677454  
2015-07-09 00:44:36 19638 [Warning] WSREP: failed to replay trx: source: f3b46697-1ff6-11e5-af61-0b245f7246eb version: 3 local: 1 state: REPLAYING flags: 129 conn\_id: 6286358 trx\_id: 366113962 seqnos (l: 6786080, g: 46677454, s: 46677452, d: 46677453, ts: 2721954221674858)  
2015-07-09 00:44:36 19638 [Warning] WSREP: Failed to apply trx 46677454 4 times  
2015-07-09 00:44:36 19638 [ERROR] WSREP: trx\_replay failed for: 6, query: void  
2015-07-09 00:44:36 19638 [ERROR] Aborting

After the node fails, it ALWAYS has to do a SST (instead of an IST). I have tried to Google around and I have found several people having the same issue, but no resolutions. Is this a configuration problem? A bug that needs to be reported?

Any help would be appreciated. Thanks in advance!