Hi,
We have a 3 node (Server version: 5.6.28-76.1-56-log,Release rel76.1, Revision f9b078d, WSREP version 25.14, wsrep_25.14) Percona XtraDB Cluster which recently crashed with the following message:
2016-06-04 18:19:13 6974 [ERROR] Slave SQL: Could not execute Delete_rows event on table Table.CDS; Cannot delete or update a parent row: a foreign key constraint fails (Table.SDS, CONSTRAINT SDS_FK2 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID)), Error_code: 1451; handler error HA_ERR_ROW_IS_REFERENCED; the event’s master log FIRST, end_log_pos 2647, Error_code: 1451
2016-06-04 18:19:13 6974 [Warning] WSREP: RBR event 23 Delete_rows apply warning: 152, 5189936
2016-06-04 18:19:13 6974 [Warning] WSREP: failed to replay trx: source: 0df79e2b-165a-11e6-b44e-76e9cbdfa4df version: 3 local: 1 state: REPLAYING flags: 1 conn_id: 1288963 trx_id: 60382186 seqnos (l: 5284194, g: 5189936, s: 5189934, d: 5189935, ts: 6916095755573834)
2016-06-04 18:19:13 6974 [Warning] WSREP: Failed to apply trx 5189936 4 times
2016-06-04 18:19:13 6974 [ERROR] WSREP: trx_replay failed for: 6, schema: table, query: (null)
2016-06-04 18:19:13 6974 [ERROR] Aborting
2016-06-04 18:19:13 6974 [Warning] WSREP: 0df79e2b sending install message failed: Resource temporarily unavailable
2016-06-04 18:19:13 6974 [Note] WSREP: view(view_id(NON_PRIM,0df79e2b,6) memb {
0df79e2b,0
} joined {
} left {
} partitioned {
14478fd2,0
36b550d6,0
})
2016-06-04 18:19:13 6974 [Note] WSREP: forgetting 36b550d6 (tcp://10.129.4.172:4567)
2016-06-04 18:19:13 6974 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2016-06-04 18:19:13 6974 [Note] WSREP: Flow-control interval: [16, 16]
2016-06-04 18:19:13 6974 [Note] WSREP: Received NON-PRIMARY.
2016-06-04 18:19:13 6974 [Note] WSREP: view(view_id(NON_PRIM,0df79e2b,7) memb {
0df79e2b,0
} joined {
} left {
} partitioned {
14478fd2,0
36b550d6,0
})
2016-06-04 18:19:13 6974 [Note] WSREP: Shifting SYNCED → OPEN (TO: 5189936)
2016-06-04 18:19:13 6974 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2016-06-04 18:19:13 6974 [Note] WSREP: Flow-control interval: [16, 16]
2016-06-04 18:19:13 6974 [Note] WSREP: Received NON-PRIMARY.
2016-06-04 18:19:15 6974 [Note] WSREP: killing local connection: 1469116
Do you have any suggestions on troubleshooting or preventing this from re-occuring?
Thank you in advance for your replies