# Percona-XtraDB-Cluster 5.7.19 Bug or Not? **URL:** \ **Category:** Percona XtraDB Cluster 5.x\ **Created:** [March 13, 2018, 2:14am UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248 "2018-03-13T02:14:39Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![mehman](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@mehman](https://forums.percona.com/u/mehman)\ **Post date:** [March 13, 2018, 2:14am UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248/1 "2018-03-13T02:14:39Z")
I have a 3-node Cluster installed on Oracle Linux Server release 7.3. On each Node RAM is 16GB. 1st Node shutdown immediately and error in log is like following 2018-03-10T17:53:34.779241Z 10 [Note] WSREP: Victim thread: THD: 1362991, mode: local, state: committing, conflict: no conflict, seqno: -1 SQL: commit 2018-03-10T17:59:14.170233Z 1369479 [Note] Aborted connection 1369479 to db: ‘shems’ user: ‘asterisk’ host: ‘10.22.12.105’ (Got an error reading communication packets) 2018-03-10T18:03:32.001458Z 1370575 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort \*\*\* Priority TRANSACTION: TRANSACTION 12735007, ACTIVE 0 sec starting index read mysql tables in use 1, locked 1 MySQL thread id 7, OS thread handle 139814477833984, query id 48121740 System lock \*\*\* Victim TRANSACTION: TRANSACTION 12735006, ACTIVE 0 sec , undo log entries 1 MySQL thread id 1362991, OS thread handle 139814137227008, query id 48121738 10.22.12.105 shems wsrep: initiating replication for write set (-1) commit \*\*\* WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1566 page no 3 n bits 88 index PRIMARY of table `shems`.`realtime_map` trx id 12735006 lock\_mode X locks rec but not gap 2018-03-10T18:05:47.369670Z 7 [Note] WSREP: --------- CONFLICT DETECTED -------- 2018-03-10T18:05:47.369680Z 7 [Note] WSREP: cluster conflict due to high priority abort for threads: 2018-03-10T18:05:47.369687Z 7 [Note] WSREP: Winning thread: THD: 7, mode: applier, state: executing, conflict: no conflict, seqno: 3779483 SQL: (null) 2018-03-10T18:05:47.369693Z 7 [Note] WSREP: Victim thread: THD: 1362991, mode: local, state: committing, conflict: no conflict, seqno: -1 SQL: commit 2018-03-10T18:22:47.005681Z 1373879 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T18:43:01.001650Z 1377351 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T18:43:01.003203Z 1377353 [Note] WSREP: MDL conflict db= table=task\_event ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T19:03:03.001551Z 1380791 [Note] WSREP: MDL conflict db= table=task\_event ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T20:00:51.686305Z 1390566 [Note] Aborted connection 1390566 to db: ‘shems’ user: ‘asterisk’ host: ‘10.22.12.105’ (Got an error reading communication packets) 2018-03-10T20:05:02.002695Z 1391429 [Note] WSREP: MDL conflict db= table=task\_event ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T20:05:02.003068Z 1391430 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T20:08:06.370495Z 1308616 [Note] Aborted connection 1308616 to db: ‘shems’ user: ‘asterisk’ host: ‘10.22.12.105’ (Got an error reading communication packets) 2018-03-10T20:45:08.784890Z 3 [Note] WSREP: MDL conflict db= table= ticket=MDL\_SHARED solved by abort 2018-03-10T20:45:09.755715Z 1398317 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T20:50:57.001104Z 1399312 [Note] WSREP: MDL conflict db= table=task\_event ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T20:50:57.001923Z 1399313 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort 2018-03-10T21:58:49.001785Z 1410960 [Note] WSREP: MDL conflict db= table=task\_event\_2 ticket=MDL\_EXCLUSIVE solved by abort \*\*\* Priority TRANSACTION: TRANSACTION 12770159, ACTIVE 0 sec starting index read mysql tables in use 1, locked 1 MySQL thread id 8, OS thread handle 139814477035264, query id 48592432 System lock \*\*\* Victim TRANSACTION: TRANSACTION 12770158, ACTIVE 0 sec fetching rows mysql tables in use 1, locked 1 MySQL thread id 1411254, OS thread handle 139814133233408, query id 48592430 event\_scheduler updating update call\_durations set call\_duration=@duration where call\_id= NAME\_CONST(‘cid’,\_utf8’1520664191.13624’ COLLATE ‘utf8\_general\_ci’) and user\_id= NAME\_CONST(‘uid’,8) \*\*\* WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 1468 page no 4 n bits 264 index GEN\_CLUST\_INDEX of table `shems`.`call_durations` trx id 12770158 lock\_mode X 2018-03-10T22:00:31.075756Z 8 [Note] WSREP: --------- CONFLICT DETECTED -------- 2018-03-10T22:00:31.075777Z 8 [Note] WSREP: cluster conflict due to high priority abort for threads: 2018-03-10T22:00:31.075828Z 8 [Note] WSREP: Winning thread: THD: 8, mode: applier, state: executing, conflict: no conflict, seqno: 3796899 SQL: (null) 2018-03-10T22:00:31.075849Z 8 [Note] WSREP: Victim thread: THD: 1411254, mode: local, state: idle, conflict: no conflict, seqno: -1 SQL: update call\_durations set call\_duration=@duration where call\_id= NAME\_CONST(‘cid’,\_utf8’1520664191.13624’ COLLATE ‘utf8\_general\_ci’) and user\_id= NAME\_CONST(‘uid’,8) 2018-03-11 02:00:31 0x7f2903bef700 InnoDB: Assertion failure in thread 139814133233408 in file lock0lock.cc line 7973 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to [http://bugs.mysql.com](http://bugs.mysql.com). InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: [url][http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html[/url]](http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html%5B/url%5D) InnoDB: about forcing recovery. 22:00:31 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. Please help us make Percona XtraDB Cluster better by reporting any bugs at [url][https://bugs.launchpad.net/percona-xtradb-cluster[/url]](https://bugs.launchpad.net/percona-xtradb-cluster%5B/url%5D) key\_buffer\_size=134217728 read\_buffer\_size=2097152 max\_used\_connections=12 max\_threads=152 thread\_count=15 connection\_count=3 It is possible that mysqld could use up to key\_buffer\_size + (read\_buffer\_size + sort\_buffer\_size)\*max\_threads = 751788 K bytes of memory Hope that’s ok; if not, decrease some variables in the equation. Thread pointer: 0x7f28802cfe00 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong… stack\_bottom = 7f2903beeabf thread\_stack 0x40000 /usr/sbin/mysqld(my\_print\_stacktrace+0x3b)[0xf3de9b] /usr/sbin/mysqld(handle\_fatal\_signal+0x471)[0x7adfc1] /lib64/libpthread.so.0(+0xf370)[0x7f2938714370] /lib64/libc.so.6(gsignal+0x37)[0x7f29367f21d7] /lib64/libc.so.6(abort+0x148)[0x7f29367f38c8] /usr/sbin/mysqld[0x77766a] /usr/sbin/mysqld[0x109158d] /usr/sbin/mysqld(\_ZN7RecLock14deadlock\_checkEP9ib\_lock\_t+0x1d)[0x10915ad] /usr/sbin/mysqld(\_ZN7RecLock12add\_to\_waitqEPK9ib\_lock\_tPK9lock\_prdt+0x82)[0x1091dc2] /usr/sbin/mysqld[0x109251f] /usr/sbin/mysqld(\_Z34lock\_clust\_rec\_read\_check\_and\_lockmPK11buf\_block\_tPKhP12dict\_index\_tPKm9lock\_modemP9que\_thr\_t+0xe5)[0x1093a85] /usr/sbin/mysqld[0x115337a] /usr/sbin/mysqld(\_Z15row\_search\_mvccPh15page\_cur\_mode\_tP14row\_prebuilt\_tmm+0x1635)[0x115b715] /usr/sbin/mysqld(\_ZN11ha\_innobase13general\_fetchEPhjj+0x1a5)[0x1041ae5] /usr/sbin/mysqld(\_ZN7handler11ha\_rnd\_nextEPh+0x177)[0x82a097] /usr/sbin/mysqld(\_Z13rr\_sequentialP11READ\_RECORD+0x20)[0xc5f250] /usr/sbin/mysqld(_Z12mysql\_updateP3THDR4ListI4ItemES4\_y15enum\_duplicatesPyS6_+0x11bf)[0xd8dd5f] /usr/sbin/mysqld(\_ZN14Sql\_cmd\_update23try\_single\_table\_updateEP3THDPb+0x1d6)[0xd904b6] /usr/sbin/mysqld(\_ZN14Sql\_cmd\_update7executeEP3THD+0x36)[0xd907f6] /usr/sbin/mysqld(\_Z21mysql\_execute\_commandP3THDb+0x3797)[0xd000c7] /usr/sbin/mysqld(\_ZN13sp\_instr\_stmt9exec\_coreEP3THDPj+0x50)[0xc7bb40] /usr/sbin/mysqld(\_ZN12sp\_lex\_instr23reset\_lex\_and\_exec\_coreEP3THDPjb+0x374)[0xc7d774] /usr/sbin/mysqld(\_ZN12sp\_lex\_instr29validate\_lex\_and\_execute\_coreEP3THDPjb+0xbb)[0xc7e15b] /usr/sbin/mysqld(\_ZN13sp\_instr\_stmt7executeEP3THDPj+0x330)[0xc7f510] /usr/sbin/mysqld(\_ZN7sp\_head7executeEP3THDb+0x53b)[0xc7737b] /usr/sbin/mysqld(\_ZN7sp\_head17execute\_procedureEP3THDP4ListI4ItemE+0x7a7)[0xc7b017] /usr/sbin/mysqld(\_ZN14Event\_job\_data7executeEP3THDb+0xabb)[0xdd9ccb] /usr/sbin/mysqld(\_ZN19Event\_worker\_thread3runEP3THDP28Event\_queue\_element\_for\_exec+0x188)[0xe95cd8] /usr/sbin/mysqld(event\_worker\_thread+0x57)[0xe95da7] /usr/sbin/mysqld(pfs\_spawn\_thread+0x1b4)[0xfc2394] /lib64/libpthread.so.0(+0x7dc5)[0x7f293870cdc5] /lib64/libc.so.6(clone+0x6d)[0x7f29368b473d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (48e8c48): update call\_durations set call\_duration=@duration where call\_id= NAME\_CONST(‘cid’,\_utf8’1520664191.13624’ COLLATE ‘utf8\_general\_ci’) and user\_id= NAME\_CONST(‘uid’,8) Connection ID (thread ID): 1411254 Status: NOT\_KILLED [COLOR=#FF0000]WHAT IS THE REASON? IS IT BUG OR NOT? ---
**Author:** ![Kenn\_Takara](https://avatars.discourse-cdn.com/v4/letter/k/b2d939/32.png) [@Kenn\_Takara](https://forums.percona.com/u/Kenn_Takara)\ **Post date:** [March 14, 2018, 6:58am UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248/2 "2018-03-14T06:58:34Z")
Anytime a crash like this happens, it’s a bug, it’s running into an assert in the code. What is the exact scenario? From the stack trace it looks like you’re calling a stored procedure from an event. Is this correct? ---
**Author:** ![Kenn\_Takara](https://avatars.discourse-cdn.com/v4/letter/k/b2d939/32.png) [@Kenn\_Takara](https://forums.percona.com/u/Kenn_Takara)\ **Post date:** [March 14, 2018, 9:16pm UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248/3 "2018-03-14T21:16:29Z")
This appears to be a bug when the event gets rolled back due to conflict. This occurs on both PXC 5.6 and 5.7. A jira bug has been created for this: [url][[PXC-2079] Crash during rollback of a stored procedure called in an event - Percona JIRA](https://jira.percona.com/browse/PXC-2079%5B/url%5D) ---
**Author:** ![mehman](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@mehman](https://forums.percona.com/u/mehman)\ **Post date:** [March 15, 2018, 12:27am UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248/4 "2018-03-15T00:27:18Z")
Yes it is an event calling procedure. Generally I think there is some problems with events in PXC 5.7. For example in my situation (3-node cluster). The Events are only enabled in 1st node by default. If 1-st node is crash then the events are not enabled in one of the other 2 nodes automatically. Even if I forced to enable in one of other 2 nodes after crash, although status of events show enabled, actually doesn’t work. Means that events are only works in 1-st node - originated node. Is it really work principle of events in PXC 5.7? If yes then it is the problem for cluster and hope to be solved in following releases or can be solved with patch or bug fixes. If there is any documentation or blog which explain events detailed, please inform me. Thanks. ---
**Author:** ![Kenn\_Takara](https://avatars.discourse-cdn.com/v4/letter/k/b2d939/32.png) [@Kenn\_Takara](https://forums.percona.com/u/Kenn_Takara)\ **Post date:** [March 15, 2018, 8:09pm UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248/5 "2018-03-15T20:09:19Z")
As far as events are concerned, PXC (galera) replication should behave similarly to asynchronous replication. This is a link to some documentation from MySQL. [https://dev.mysql.com/doc/refman/5.7/en/replication-features-invoked.html](https://dev.mysql.com/doc/refman/5.7/en/replication-features-invoked.html) I’ve tested the events and they seem to work on the other nodes (after they are enabled and event-scheduler is set to ON) and as long as the event action is not rolled back due to a BF abort. There is no automatic failover of the events in the case a node goes down. ---
**Author:** ![mehman](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@mehman](https://forums.percona.com/u/mehman)\ **Post date:** [March 16, 2018, 1:41am UTC](https://forums.percona.com/t/percona-xtradb-cluster-5-7-19-bug-or-not/6248/6 "2018-03-16T01:41:57Z")
Thanks Kenn Takara, It is clear now. In my.cnf event-scheduler configured as OFF. I forget about it. So it doesn’t work.