Open Transactions After wsrep-osu-method=NBO

Hi all,

I’m using wsrep_OSU_method NBO on the 3 nodes cluster. When the DDL operation completed one of the nodes on the cluster, on the other nodes I noticed that these nodes have open transactions. Because of this binlog files are not relocated on these nodes.

I noticed this from binlog file sizes. Here is the latest binlog file sizes for all nodes. BTW, I ran the DDL command with NBO method on node2.

node1 - 9.2G Dec 12 12:39 binlog.000254
node2 - 9.1M Dec 12 12:40 binlog.000238
node3 - 8.5G Dec 12 12:40 binlog.000416

Additionaly, today I need to restore the latest xtrabackup taken from node3 (the problematic node). I checked the backup log, it seems it finished successfully. But when I try to start the restore mysql node I get the following error. Because of this, I can’t start the MySQL service right now.

2023-12-12T09:25:53.810621Z 0 [ERROR] [MY-013909] [Server] Found invalid event sequence while recovering from binary log file ‘./binlog.000416’, between positions 109079998 and 109080475: Xid_log_event holds an invalid XID. The recovery process was stopped early and no transaction was recovered. Side effects may be transactions in an inconsistent state between the binary log and the storage engines, or transactions kept by storage engines in a prepared state (possibly holding locks). Either fix the issues with the binary log or, to release possibly acquired locks, disable the binary log during server recovery. Note that disabling the binary log may lead to loss of transactions that were already acknowledged as successful to client connections and may have been replicated to other servers in the topology.

Is there any idea?