We are running into a new issue with our Percona XtraDB cluster which is comprised of 4 nodes and an arbitrator in AWS as EC2 instances separated into two availability zones in the N. Virginia region. We are running version Ver 8.0.36-28.1.
It seems like it might be related to a bug posted here (PXC-4385: MDL conflict leads to locked nodes · percona/percona-xtradb-cluster@fd560ad · GitHub) but not 100% sure so figured I would post here for feedback. The cluster had very light activity taking place prior, during and after the event.
It is MDL related and here is some information from the logs.
2024-06-28T15:38:23.081089Z 61968155 [Note] [MY-000000] [WSREP] MDL BF-BF conflict
schema:
request: (thd-tid:61968155 seqno:296358467 exec-mode:high priority, query-state:exec, conflict-state:executing)
cmd-code:3 61 query:BEGIN)
granted: (thd-tid:4040320 seqno:296358466 exec-mode:toi, query-state:exec, conflict-state:committed)
cmd-code:3 34 query:GRANT SELECT, INSERT, UPDATE, DELETE ON database.table TO ‘user’@‘%’)
2024-06-28T15:38:23.081546Z 61968155 [Note] [MY-000000] [WSREP] MDL ticket: type: exclusive, space: UNKNOWN, db: , name:
2024-06-28T15:38:23.081561Z 61968155 [ERROR] [MY-010119] [Server] Aborting
2024-06-28T15:38:23.082097Z 61968155 [Note] [MY-000000] [WSREP] Initiating SST cancellation
2024-06-28T15:38:25.083840Z 61968155 [Warning] [MY-000000] [Server] /usr/sbin/mysqld: Forcing close of thread 61968160 user: ‘’
2024-06-28T15:38:25.083918Z 61968155 [Warning] [MY-000000] [Server] /usr/sbin/mysqld: Forcing close of thread 61968168 user: ‘’
2024-06-28T15:38:25.083949Z 61968155 [Warning] [MY-000000] [Server] /usr/sbin/mysqld: Forcing close of thread 61968184 user: ‘’
2024-06-28T15:38:25.083965Z 61968155 [Warning] [MY-000000] [Server] /usr/sbin/mysqld: Forcing close of thread 61968192 user: ‘’
SST was in process and there are no other references to SST in the log for the 28th as I triple-checked that. So I don’t understand the [WSREP] Initiating SST cancellation statement at all in this context.
Any thoughts, ideas, or recommendations would be appreciated.