# Percona xtradb cluster crash: HA\_ERR\_ROW\_IS\_REFERENCED

**URL:** <https://forums.percona.com/t/percona-xtradb-cluster-crash-ha-err-row-is-referenced/6165>\
**Category:** Percona XtraDB Cluster 5.x\
**Created:** [January 30, 2018, 10:28am UTC](https://forums.percona.com/t/percona-xtradb-cluster-crash-ha-err-row-is-referenced/6165 "2018-01-30T10:28:19Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jjost](https://avatars.discourse-cdn.com/v4/letter/j/51bf81/32.png) [@jjost](https://forums.percona.com/u/jjost)\
**Post date:** [January 30, 2018, 10:28am UTC](https://forums.percona.com/t/percona-xtradb-cluster-crash-ha-err-row-is-referenced/6165/1 "2018-01-30T10:28:19Z")

</div>

Hi,

We migrated a series of 100G Databases into different Galera Instances.Its running fine so far. I do have 5-6 years experience with Galera.  
The cluster has 5 Nodes and the Loadbalancers always targets 1 Node for Read/Write and only takes the second(third…) when the primary node is down (or reports wsrep\_ready off).

Version is the same on all servers:  
Server version: 5.6.37-82.2-56-log Percona XtraDB Cluster (GPL), Release rel82.2, Revision 114f2f2, WSREP version 26.21, wsrep\_26.21

In the night, always roughly at the same time, all 4 non-active nodes drop-out of the cluster at the same time with this error:  
2018-01-29 01:26:56 33304 [ERROR] Slave SQL: Could not execute Delete\_rows event on table storedb.object; Cannot delete or update a parent row: a foreign key constraint fails (`storedb`.`object`, CONSTRAINT `fk_object_object` FOREIGN KEY (`parentid`) REFERENCES `object` (`objectid`)), Error\_code: 1451; handler error HA\_ERR\_ROW\_IS\_REFERENCED; the event’s master log FIRST, end\_log\_pos 250, Error\_code: 1451  
2018-01-29 01:26:56 33304 [Warning] WSREP: RBR event 3 Delete\_rows apply warning: 152, 3097067  
2018-01-29 01:26:56 33304 [Warning] WSREP: Failed to apply app buffer: seqno: 3097067, status: 1

The table is looking like this:  
| object | CREATE TABLE `object` (  
`objectid` int(11) NOT NULL AUTO\_INCREMENT,  
`classid` int(11) NOT NULL,  
`parentid` int(11) DEFAULT NULL,  
`alias` varchar(255) COLLATE utf8\_bin NOT NULL,  
`siteid` int(11) NOT NULL,  
PRIMARY KEY (`objectid`),  
UNIQUE KEY `u_object_guid` (`guid`),  
UNIQUE KEY `u_object_alias` (`parentid`,`alias`),  
KEY `i_object_class` (`classid`),  
KEY `i_object_siteid_classid_alias` (`siteid`,`classid`,`alias`),  
CONSTRAINT `fk_object_object` FOREIGN KEY (`parentid`) REFERENCES `object` (`objectid`),  
CONSTRAINT `fk_object_object_1` FOREIGN KEY (`siteid`) REFERENCES `object` (`objectid`)  
) ENGINE=InnoDB AUTO\_INCREMENT= DEFAULT CHARSET=utf8 COLLATE=utf8\_bin |

The Query that is generating the error is _really_ easy:  
DELETE FROM `storedb`.`object` WHERE objectid=‘highnumber’;

I saved the datadir from the moment before the cluster crashed (just kept one node out of the cluster after the crash and started it without wsrep provider to have a state to play with).  
I can issue the query above with no problem. It deletes the row.

A corresponding GRA.log file is generated but mysqbinlog complains that it cant read the event:  
ERROR: Error in Log\_event::read\_log\_event(): ‘Found invalid event in binary log’, data\_len: 105, event\_type: 32

I am running weekly consistency checks (percona pt-table-checksum) and they report no inconsistency.

Any idea what is happening here?

Greetings  
julian

---

<div class="post-metadata">

**Author:** ![jjost](https://avatars.discourse-cdn.com/v4/letter/j/51bf81/32.png) [@jjost](https://forums.percona.com/u/jjost)\
**Post date:** [February 1, 2018, 5:35am UTC](https://forums.percona.com/t/percona-xtradb-cluster-crash-ha-err-row-is-referenced/6165/2 "2018-02-01T05:35:29Z")

</div>

Hey,

as an update: It is NOT happening when wsrep\_slave\_threads are set to 1.  
But this is not a solution as it slows down the Cluster.

---

<div class="post-metadata">

**Author:** ![krunalbauskar](https://avatars.discourse-cdn.com/v4/letter/k/439d5e/32.png) [@krunalbauskar](https://forums.percona.com/u/krunalbauskar)\
**Post date:** [February 5, 2018, 6:07am UTC](https://forums.percona.com/t/percona-xtradb-cluster-crash-ha-err-row-is-referenced/6165/3 "2018-02-05T06:07:11Z")

</div>

Hi,

Do you have reproducible test-case? I tried looking at the output but don’t foresee any immediate clue.

---

<div class="post-metadata">

**Author:** ![jjost](https://avatars.discourse-cdn.com/v4/letter/j/51bf81/32.png) [@jjost](https://forums.percona.com/u/jjost)\
**Post date:** [February 15, 2018, 10:36am UTC](https://forums.percona.com/t/percona-xtradb-cluster-crash-ha-err-row-is-referenced/6165/4 "2018-02-15T10:36:21Z")

</div>

Hi,

I am still working on that. It seems quite hard to trigger (with my limited skills in programming). With bash, mysql binary and mysqlimport, even in parrallel i can’t trigger it. I _do_ have code inside a big application i can run and it crashes in a few seconds but that is nothing i can provide so far.

Greetings