After upgrade to 8.0.30 errors: MDL conflict db= table= ticket=10 solved by abort

Hi,
we upgraded our some pxc 3-node clusters from 8.0.27, 8.0.29 to 8.0.30-22.md
After that in error.log we are getting a lot of errors:
2023-01-16T09:22:01.962511-00:00 161616 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-01-16T09:22:01.962661-00:00 161617 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-16T10:22:02.064383-00:00 162676 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-01-16T10:22:02.064711-00:00 162677 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-16T11:59:55.946132-00:00 163948 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-16T13:35:52.337620-00:00 165300 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-01-16T13:35:52.337849-00:00 165301 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort

Is there any way to see reasons for this?
we have wsrep_log_conflicts=ON
I tried:
set global wsrep_debug=1;
set global wsrep_provider_options=“cert.log_conflicts=ON”;
didn’t help, still a lot of errors witout table name:
[Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort

P.S. before 8.0.30 we have sertification errors like:
TRANSACTION 59238092, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT
MySQL thread id 18, OS thread handle 140076065289984, query id 32898852 Applying batch of row changes (update)
TRANSACTION 59238089, ACTIVE 0 sec
, undo log entries 1
MySQL thread id 161585, OS thread handle 140059226334976, query id 32898850 10.1.2.158 grafana wsrep: replicating and certifying write set(-1)
COMMIT
2023-01-16T09:19:16.667847-00:00 18 [Note] [MY-000000] [WSREP] --------- CONFLICT DETECTED --------
2023-01-16T09:19:16.667889-00:00 18 [Note] [MY-000000] [WSREP] cluster conflict due to high priority abort for threads:

2023-01-16T09:19:16.667917-00:00 18 [Note] [MY-000000] [WSREP] Winning thread:
THD: 18, mode: high priority, state: exec, conflict: executing, seqno: 5353972
SQL: (null)

2023-01-16T09:19:16.667943-00:00 18 [Note] [MY-000000] [WSREP] Victim thread:
THD: 161585, mode: local, state: exec, conflict: certifying, seqno: -1
SQL: COMMIT

Now we have some of the same error and in addition these:
[Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort

2 Likes

can you please check if you are doing any DDL with wsrep_OSU_method=RSU;

Also during the DDL your selects must be throwing errors like below

ERROR 1213 (40001): WSREP detected deadlock/conflict and aborted the transaction. Try restarting the transaction

1 Like

No, we are not doing any DDL. Just the same application work as in previous pxc versions 8.0.27 and 8.0.29

mysql> SHOW VARIABLES LIKE 'wsrep_OSU_method';
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| wsrep_OSU_method | TOI   |
+------------------+-------+
1 Like

I’m getting the same error messages on a clean install, bootstrapped first node, after import from mysqldump.

They are none stop every few seconds.

2023-01-28T05:32:00.370233Z 970277 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-28T05:32:00.370284Z 970276 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-28T05:32:00.370372Z 970280 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-28T05:32:00.370513Z 970280 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-01-28T05:32:01.641143Z 970290 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-01-28T05:32:13.726154Z 970328 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-01-28T05:32:13.726306Z 970327 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort

Any suggestions on determining the cause of these messages? This error is pretty unhelpful. DB appears to work fine but seems a bit slow.

Thank you

Hi @slevytam

Are you writing on all the nodes or just on one? Usually these are normal conflicts when multiple nodes receive writes. This situation can lead to concurrency conflicts that are resolved by the cluster and, much more important, that need to be handled properly by the application.

Also, what is the value of wsrep_log_conflicts?
https://galeracluster.com/library/documentation/mysql-wsrep-options.html#wsrep-log-conflicts

Thanks,

Pep

Thanks for your reply.

wsrep_log_conflicts = ON

This was happening on the first node before i added subsequent nodes. I’m not sure how there could be constant concurrency conflicts with just one node up.

Your assistance is appreciated!

What is the version of PXC that you installed? The same as in the original post? Please also provide additional information about the operating system version.

Yes, its the same version as the original post.

Running percona-xtradb-cluster-garbd.x86_64 (8.0.30-22.1.el9) on clean install of CentOS Stream 9.

Thanks

Hi,
I’m getting the same conflict message from all nodes even if only one node is used for write operation from the load balancer.

Our xtradb cluster distribution is 8.0.30-22-1.focal on ubuntu 20.04.5.

2023-02-01T12:16:51.727756Z 22118 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-02-01T12:17:05.170599Z 22172 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-01T12:18:49.230556Z 22638 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-02-01T12:20:53.232776Z 23164 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-01T12:21:15.901774Z 23264 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-01T12:21:15.902001Z 23265 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-01T12:21:31.194756Z 23339 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-02-01T12:23:10.990283Z 23790 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-01T12:24:19.060378Z 24109 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-02-01T12:27:10.244016Z 24893 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-01T12:27:10.244308Z 24894 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort

My OS:
uname -a
Linux prod-db2 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21) x86_64 GNU/Linux

Hello
We have the same problem on an updated node (versions 8.0.30-22.1.el8) of a 3-node pcx cluster. Two remaining non-updated nodes (versions 8.0.26-16.1.el8) do not write errors to the log. OS oracle linux 8 (5.4.17-2136.305.5.4.el8uek.x86_64)

I’m seeing this exact same behavior on Ubuntu 22.04. Only the host being written to is spamming:

After a day or two that node crashes, and it begins on the next host.

Hi,

Having same issue. Getting these logs in writer node error logs.

2023-02-09T14:12:54.546726+05:00 1394 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-09T14:13:13.694776+05:00 1413 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-09T14:13:34.130040+05:00 1427 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=1 solved by abort
2023-02-09T14:13:34.130162+05:00 1429 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-09T14:13:44.330279+05:00 1442 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort
2023-02-09T14:14:13.664067+05:00 1468 [Note] [MY-000000] [WSREP] MDL conflict db= table= ticket=10 solved by abort

Using ubuntu 20.04 and percona-xtradb-cluster 1:8.0.30-22-1.focal

I have upgraded from percona-xtradb-cluster 1:8.0.25-15-1.focal

Ammad Ali

I have got only a one-time crash, could you please share your crash log to compare?

Ugh another crash

1 Like

1 Like

@Percona_Staff Could we please get some insight on this issue?

1 Like

Hi @Chris_Bunge
Are you able to reproduce with a test case?

It takes about 48 hours but always reproduces in my lab. I had restored a db from 5.7 that ran for years if that is applicable.

Unsure how to reproduce in your environment atm.