PXC 8.0.46-38 remains in Donor/Desynced after mydumper backup

MySQL & MariaDB Percona XtraDB Cluster
1

Since upgrading our Percona XtraDB Cluster to 8.0.46-38, one node remains stuck in Donor/Desynced after a mydumper backup.

This backup setup had been working for years before the upgrade.

Our original backup script was:

mysql --defaults-extra-file="${CREDFILE}" \
    --execute "SET GLOBAL wsrep_desync = ON"

mydumper \
    --defaults-file="${CREDFILE}" \
    --use-savepoints \
    -t 2 \
    -c ZSTD \
    -o "${DESTDIR}/0"

mysql --defaults-extra-file="${CREDFILE}" \
    --execute "SET GLOBAL wsrep_desync = OFF"

After the backup, wsrep_desync is correctly set to OFF, but the node does not return to Synced.

The status then looks like this:

wsrep_desync              OFF
wsrep_desync_count        1
wsrep_local_state         2
wsrep_local_state_comment Donor/Desynced
wsrep_local_recv_queue    0
wsrep_connected           ON
wsrep_ready               ON

Running SET GLOBAL wsrep_desync = OFF again only gives: wsrep_desync is already OFF. The node only returns to Synced after restarting MySQL.

We tested a few different scenarios.

First, we tested wsrep_desync without mydumper:

SET GLOBAL wsrep_desync = ON;
-- wait about 10 minutes
SET GLOBAL wsrep_desync = OFF;

That works normally. wsrep_desync_count returns to 0 and the node becomes Synced again. With the original backup script, the values change like this:

Before backup:
wsrep_desync=0
wsrep_desync_count=0
Synced

During mydumper:
wsrep_desync=1
wsrep_desync_count=2
Donor/Desynced

After SET GLOBAL wsrep_desync=OFF:
wsrep_desync=0
wsrep_desync_count=1
Donor/Desynced

We then ran mydumper without setting wsrep_desync ourselves. The result was:

Before mydumper:
wsrep_desync=0
wsrep_desync_count=0
Synced

After mydumper started:
wsrep_desync=0
wsrep_desync_count=1
Donor/Desynced

The counter stayed at 1 after mydumper completed.

So mydumper alone is enough to trigger the problem. The explicit wsrep_desync commands are not the cause. We first reproduced this with: mydumper v0.16.1-3. We then upgraded to the latest mydumper version, but the result was the same.

The problem does not occur when using:

--sync-thread-lock-mode=SAFE_NO_LOCK
--no-backup-locks

It also does not occur with:

--sync-thread-lock-mode=GTID
--no-backup-locks

With both settings, the backup completes and the node returns to:

wsrep_desync=OFF
wsrep_desync_count=0
wsrep_local_state_comment=Synced

At this point, it looks like the issue is in the interaction between PXC 8.0.46-38 and the default lock path selected by mydumper with sync-thread-lock-mode=FTWRL.

My assumption is that mydumper’s use of FTWRL causes PXC to create an internal desync reference. After the lock is released, that reference is not removed, leaving the node stuck with:

wsrep_desync=OFF
wsrep_desync_count=1
Donor/Desynced

The issue appeared immediately after upgrading to PXC 8.0.46-38. Since that release includes changes to FTWRL handling on backup nodes, including PXC-4799, it seems possible that the change is related.

Has anyone else seen this with PXC 8.0.46-38?

2

@DeBuXer I was also able to reproduce it. Will check this internally with Dev and then update you.

SET GLOBAL wsrep_desync = ON

node1 [localhost:30647] {root} ((none)) > show global status like '%wsrep_local_state_comment%';
+---------------------------+----------------+
| Variable_name             | Value          |
+---------------------------+----------------+
| wsrep_local_state_comment | Donor/Desynced |
+---------------------------+------

mydumper --threads=4 --user=root --password=msandbox --host=127.0.0.1 --port=30647 --trx-tables --events --routines --triggers --compress --outputdir backup/ --logfile mydumper.out --verbose=3

mydumper.out:

2026-08-12 05:41:29 [INFO] - Releasing DDL lock
2026-08-12 05:41:29 [INFO] - UNLOCK INSTANCE: OK
2026-08-12 05:41:29 [INFO] - Queue count: 0 0 0 0 0
2026-08-12 05:41:29 [INFO] - Main connection closed
2026-08-12 05:41:29 [INFO] - Finished dump at: 2026-08-12 05:41:29

SET GLOBAL wsrep_desync = OFF

node1 [localhost:30647] {root} ((none)) >  show global status like '%wsrep_local_state_comment%';
+---------------------------+----------------+
| Variable_name             | Value          |
+---------------------------+----------------+
| wsrep_local_state_comment | Donor/Desynced |
+---------------------------+----------------+
1 row in set (0.01 sec)

node1 [localhost:30647] {root} ((none)) > show global status like '%desync%';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_desync_count | 1     |
+--------------------+-------+
1 row in set (0.00 sec)
3

@DeBuXer

Just a FYI,

I have raised the Jira - https://perconadev.atlassian.net/browse/PXC-5307 internally regarding this issue. Rest, you can monitor the URL for any further updates.

4

Hello

I update yesterday to latest version and i have exactly same bug this morning.

I add a “stop / start mysql service” on my backup node in my script.

Yathus

5

Hi @DeBuXer,

I believe I can narrow it down a bit. FTWRL alone is fine, LOCK INSTANCE FOR BACKUP alone is fine, the problem is the two together. When FTWRL is released while the backup lock is still held, the node is not put back to Synced. Same four statements mydumper sends, by hand in one session:

mysql> LOCK INSTANCE FOR BACKUP;
mysql> FLUSH TABLES WITH READ LOCK;
mysql> show global status like 'wsrep_desync_count';   -- 1, Donor/Desynced
mysql> UNLOCK TABLES;
mysql> show global status like 'wsrep_desync_count';   -- still 1
mysql> UNLOCK INSTANCE;                                 -- still 1

FTWRL alone goes 0 → 1 → 0 and the node is Synced again after UNLOCK TABLES, so the counter itself works.

For the workaround, you don’t need to give up FTWRL. --skip-ddl-locks alone is enough, mydumper then sends only FTWRL and the node returns to Synced. --no-backup-locks is not enough, with it mydumper still sends LOCK INSTANCE FOR BACKUP, that is probably why you had to combine it with SAFE_NO_LOCK or GTID.