# Delay in user synchronization across Percona XtraDB Cluster nodes **URL:** \ **Category:** Percona XtraDB Cluster 8.x\ **Created:** [October 27, 2025, 10:25am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626 "2025-10-27T10:25:31Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![TAndreotti](https://avatars.discourse-cdn.com/v4/letter/t/8baadc/32.png) [@TAndreotti](https://forums.percona.com/u/TAndreotti)\ **Post date:** [October 27, 2025, 10:25am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/1 "2025-10-27T10:25:31Z")
Hi everyone, I’m currently managing a Percona XtraDB Cluster (PXC) environment with the following setup: - **Cluster version:** 8.0.43-34.1 - Percona XtraDB Cluster (GPL), Release rel34, Revision 0682ba7, WSREP version 26.1.4.3 - **Topology:** 2-node master-master cluster + 1 arbitrator. Our database is accessed by two types of servers: 1. **“Front” servers:** Originally connected to Node 1 via HAProxy, with automatic failover to Node 2 if Node 1 crashes. Recently, some of these servers were reconfigured to use HAProxy for load balancing between both nodes. 2. **“Management” server:** Directly connected to Node 1 only. We have a script on the management server that: - Creates a new database in the cluster. - Creates a MySQL user with access only to that database. - Triggers an API call on a front server, which uses the newly created user to access the database. **Issue:** Between the user creation and the API call, there is a delay of over 1 second. Recently, we encountered a bug where the API call was routed to Node 2, which returned an error stating the user did not exist. By the time we received the error and retried manually, the user had synchronized. **Questions:** - Why isn’t user synchronization instantaneous across nodes? - How and where is user synchronization configured in PXC? - What are the limitations or key points to be aware of regarding user synchronization? I didn’t set up the cluster myself, so I’m not entirely sure how everything is configured, but I can access the servers if more details are needed. Thanks in advance for your insights! ---
**Author:** ![Yunus](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/yunus/32/6430_2.png) [@Yunus](https://forums.percona.com/u/Yunus)\ **Post date:** [October 28, 2025, 4:14am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/2 "2025-10-28T04:14:45Z")
Could you share your wsrep configuration? ```auto show global variables like '%wsrep%'; show global status like '%wsrep%'; ``` ---
**Author:** ![TAndreotti](https://avatars.discourse-cdn.com/v4/letter/t/8baadc/32.png) [@TAndreotti](https://forums.percona.com/u/TAndreotti)\ **Post date:** [October 28, 2025, 8:52am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/3 "2025-10-28T08:52:53Z")
Yep 🙂 `show global variables like '%wsrep%';` | Variable\_name | Value | | --- | --- | | wsrep\_applier\_FK\_checks | ON | | wsrep\_applier\_threads | 48 | | wsrep\_applier\_UK\_checks | OFF | | wsrep\_auto\_increment\_control | ON | | wsrep\_causal\_reads | OFF | | wsrep\_certification\_rules | strict | | wsrep\_certify\_nonPK | ON | | wsrep\_cluster\_address | gcomm://10.xx.xx.3,10.xx.xx.4 | | wsrep\_cluster\_name | cluster\_xxxxxxxxx\_01 | | wsrep\_data\_home\_dir | /var/lib/mysql/ | | wsrep\_dbug\_option | | | wsrep\_debug | NONE | | wsrep\_desync | OFF | | wsrep\_dirty\_reads | OFF | | wsrep\_disk\_pages\_encrypt | NONE | | wsrep\_gcache\_encrypt | NONE | | wsrep\_ignore\_apply\_errors | 0 | | wsrep\_load\_data\_splitting | OFF | | wsrep\_log\_conflicts | ON | | wsrep\_max\_ws\_rows | 0 | | wsrep\_max\_ws\_size | 2147483647 | | wsrep\_min\_log\_verbosity | 3 | | wsrep\_mode | | | wsrep\_node\_address | 10.xx.xx.4 | | wsrep\_node\_incoming\_address | AUTO | | wsrep\_node\_name | xxxx-04 | | wsrep\_notify\_cmd | | | wsrep\_OSU\_method | TOI | | wsrep\_provider | /usr/lib/galera4/libgalera\_smm.so | | wsrep\_provider\_options | allocator.disk\_pages\_encryption = no; allocator.encryption\_cache\_page\_size = 32K; allocator.encryption\_cache\_size = 16777216; base\_dir = /var/lib/mysql/; base\_host = 10.xx.xx.4; base\_port = 4567; cert.log\_conflicts = no; cert.optimistic\_pa = no; debug = no; evs.auto\_evict = 0; evs.causal\_keepalive\_period = PT1S; evs.debug\_log\_mask = 0x1; evs.delay\_margin = PT1S; evs.delayed\_keep\_period = PT30S; evs.inactive\_check\_period = PT0.5S; evs.inactive\_timeout = PT15S; evs.info\_log\_mask = 0; evs.install\_timeout = PT7.5S; evs.join\_retrans\_period = PT1S; evs.keepalive\_period = PT1S; evs.max\_install\_timeouts = 3; evs.send\_window = 10; evs.stats\_report\_period = PT1M; evs.suspect\_timeout = PT5S; evs.use\_aggregate = true; evs.user\_send\_window = 4; evs.version = 1; evs.view\_forget\_timeout = P1D; gcache.dir = /var/lib/mysql/; gcache.encryption = no; gcache.encryption\_cache\_page\_size = 32K; gcache.encryption\_cache\_size = 16777216; gcache.freeze\_purge\_at\_seqno = -1; gcache.keep\_pages\_count = 0; gcache.keep\_pages\_size = 0; gcache.mem\_size = 0; gcache.name = galera.cache; gcache.page\_size = 128M; gcache.recover = yes; gcache.size = 128M; gcomm.thread\_prio = ; gcs.fc\_auto\_evict\_threshold = 0.75; gcs.fc\_auto\_evict\_window = 0; gcs.fc\_debug = 0; gcs.fc\_factor = 1.0; gcs.fc\_limit = 100; gcs.fc\_master\_slave = no; gcs.fc\_single\_primary = no; gcs.max\_packet\_size = 64500; gcs.max\_throttle = 0.25; gcs.recv\_q\_hard\_limit = 9223372036854775807; gcs.recv\_q\_soft\_limit = 0.25; gcs.sync\_donor = no; gmcast.listen\_addr = tcp://0.0.0.0:4567; gmcast.mcast\_addr = ; gmcast.mcast\_ttl = 1; gmcast.peer\_timeout = PT3S; gmcast.segment = 0; gmcast.time\_wait = PT5S; gmcast.version = 0; ist.recv\_addr = 10.xx.xx.4; pc.announce\_timeout = PT3S; pc.checksum = false; pc.ignore\_quorum = false; pc.ignore\_sb = false; pc.linger = PT20S; pc.npvo = false; pc.recovery = true; pc.version = 0; pc.wait\_prim = true; pc.wait\_prim\_timeout = PT30S; pc.wait\_restored\_prim\_timeout = PT0S; pc.weight = 1; protonet.backend = asio; protonet.version = 0; repl.causal\_read\_timeout = PT30S; repl.commit\_order = 3; repl.key\_format = FLAT8; repl.max\_ws\_size = 2147483647; repl.proto\_max = 11; socket.checksum = 2; socket.recv\_buf\_size = auto; socket.send\_buf\_size = auto; | | wsrep\_recover | OFF | | wsrep\_reject\_queries | NONE | | wsrep\_replicate\_myisam | OFF | | wsrep\_restart\_replica | OFF | | wsrep\_restart\_slave | OFF | | wsrep\_retry\_autocommit | 1 | | wsrep\_RSU\_commit\_timeout | 5000 | | wsrep\_slave\_FK\_checks | ON | | wsrep\_slave\_threads | 48 | | wsrep\_slave\_UK\_checks | OFF | | wsrep\_SR\_store | table | | wsrep\_sst\_allowed\_methods | xtrabackup-v2,clone | | wsrep\_sst\_donor | | | wsrep\_sst\_donor\_rejects\_queries | OFF | | wsrep\_sst\_method | xtrabackup-v2 | | wsrep\_sst\_receive\_address | AUTO | | wsrep\_start\_position | c6f4c786-00be-11f0-8ecf-a2d19a721caa:112038713 | | wsrep\_sync\_wait | 0 | | wsrep\_trx\_fragment\_size | 0 | | wsrep\_trx\_fragment\_unit | bytes | | wsrep\_use\_async\_monitor | ON | `show global status like '%wsrep%';` | Variable\_name | Value | | --- | --- | | wsrep\_local\_state\_uuid | c6f4c786-00be-11f0-8ecf-a2d19a721caa | | wsrep\_protocol\_version | 11 | | wsrep\_last\_applied | 115069889 | | wsrep\_protocol\_application | 4 | | wsrep\_protocol\_replicator | 11 | | wsrep\_protocol\_GCS | 5 | | wsrep\_last\_committed | 115069889 | | wsrep\_monitor\_status (L/A/C) | [(3072710, 3072710), (115069889, 115069889), (115069889, 115069889)] | | wsrep\_replicated | 719971 | | wsrep\_replicated\_bytes | 1014330176 | | wsrep\_repl\_keys | 7009970 | | wsrep\_repl\_keys\_bytes | 73362608 | | wsrep\_repl\_data\_bytes | 892398608 | | wsrep\_repl\_other\_bytes | 0 | | wsrep\_received | 2352690 | | wsrep\_received\_bytes | 3357110165 | | wsrep\_local\_commits | 719898 | | wsrep\_local\_cert\_failures | 64 | | wsrep\_local\_replays | 1 | | wsrep\_local\_send\_queue | 0 | | wsrep\_local\_send\_queue\_max | 12 | | wsrep\_local\_send\_queue\_min | 0 | | wsrep\_local\_send\_queue\_avg | 0.000345927 | | wsrep\_local\_recv\_queue | 0 | | wsrep\_local\_recv\_queue\_max | 58 | | wsrep\_local\_recv\_queue\_min | 0 | | wsrep\_local\_recv\_queue\_avg | 0.00685131 | | wsrep\_local\_cached\_downto | 114954454 | | wsrep\_flow\_control\_paused\_ns | 446409 | | wsrep\_flow\_control\_paused | 7.2023e-10 | | wsrep\_flow\_control\_sent | 0 | | wsrep\_flow\_control\_recv | 1 | | wsrep\_flow\_control\_active | false | | wsrep\_flow\_control\_requested | false | | wsrep\_flow\_control\_interval | [141, 141] | | wsrep\_flow\_control\_interval\_low | 141 | | wsrep\_flow\_control\_interval\_high | 141 | | wsrep\_flow\_control\_status | OFF | | wsrep\_cert\_deps\_distance | 1.03947 | | wsrep\_apply\_oooe | 0.0242157 | | wsrep\_apply\_oool | 0.000235552 | | wsrep\_apply\_window | 1.08584 | | wsrep\_apply\_waits | 184710 | | wsrep\_commit\_oooe | 0 | | wsrep\_commit\_oool | 0 | | wsrep\_commit\_window | 1.02072 | | wsrep\_local\_state | 4 | | wsrep\_local\_state\_comment | Synced | | wsrep\_cert\_index\_size | 230 | | wsrep\_cert\_bucket\_count | 85229 | | wsrep\_gcache\_pool\_size | 134219032 | | wsrep\_causal\_reads | 64 | | wsrep\_cert\_interval | 2033.03 | | wsrep\_open\_transactions | 0 | | wsrep\_open\_connections | 0 | | wsrep\_ist\_receive\_status | | | wsrep\_ist\_receive\_seqno\_start | 0 | | wsrep\_ist\_receive\_seqno\_current | 0 | | wsrep\_ist\_receive\_seqno\_end | 0 | | wsrep\_incoming\_addresses | 10.xx.xx.4:3306,10.xx.xx.3:3306 | | wsrep\_cluster\_weight | 3 | | wsrep\_desync\_count | 0 | | wsrep\_evs\_delayed | | | wsrep\_evs\_evict\_list | | | wsrep\_evs\_repl\_latency | 0.000368835/0.000548454/0.000815358/0.000100379/22 | | wsrep\_evs\_state | OPERATIONAL | | wsrep\_gcomm\_uuid | 06cee51f-ae37-11f0-a011-2f2d9acb8a6b | | wsrep\_gmcast\_segment | 0 | | wsrep\_cluster\_capabilities | | | wsrep\_cluster\_conf\_id | 37 | | wsrep\_cluster\_size | 3 | | wsrep\_cluster\_state\_uuid | c6f4c786-00be-11f0-8ecf-a2d19a721caa | | wsrep\_cluster\_status | Primary | | wsrep\_connected | ON | | wsrep\_local\_bf\_aborts | 21 | | wsrep\_local\_index | 0 | | wsrep\_provider\_capabilities | :MULTI\_MASTER:CERTIFICATION:PARALLEL\_APPLYING:TRX\_REPLAY:ISOLATION:PAUSE:CAUSAL\_READS:INCREMENTAL\_WRITESET:UNORDERED:PREORDERED:STREAMING:NBO: | | wsrep\_provider\_name | Galera | | wsrep\_provider\_vendor | Codership Oy \ (modified by Percona \<[https://percona.com/>\](https://percona.com/>%5C)) | | wsrep\_provider\_version | 4.23(cb05b32) | | wsrep\_ready | ON | | wsrep\_thread\_count | 49 | ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [October 29, 2025, 12:05am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/4 "2025-10-29T00:05:07Z")
> [@TAndreotti](#): > > Why isn’t user synchronization instantaneous across nodes? It should be. CREATE USER, and other DCL are replicated synchronously. What’s in your error logs on both nodes during user creation? I see some flow control time; are you monitoring this? What’s your TPS/QPS? ---
**Author:** ![Yunus](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/yunus/32/6430_2.png) [@Yunus](https://forums.percona.com/u/Yunus)\ **Post date:** [October 29, 2025, 1:21am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/5 "2025-10-29T01:21:00Z")
Chceking with error log and flow control, additionally you can define in your API to set wsrep\_sync\_wait = 1 on session level if you dont want globally to have reads to be done when synced. > **[Percona XtraDB Cluster - Index of wsrep system variables](https://docs.percona.com/percona-xtradb-cluster/8.4/wsrep-system-index.html?h=wsrep_sync_wait#wsrep_sync_wait)** > > Percona XtraDB Cluster introduces a number of MySQL system variables > related to write-set replication. ---
**Author:** ![TAndreotti](https://avatars.discourse-cdn.com/v4/letter/t/8baadc/32.png) [@TAndreotti](https://forums.percona.com/u/TAndreotti)\ **Post date:** [October 29, 2025, 9:12am UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/6 "2025-10-29T09:12:01Z")
Around 1.5k QPS on the cluster… _(We have over 1,000 databases and users in the cluster.)_ I haven’t encountered any real errors. In this case the first call of the API throw : `SQLSTATE[HY000] [1045] Access denied for user 'xxxxxxxx'@'10.xxx.xxx.2' (using password: YES)` This hasn’t happened again since 🤷‍♂️ , but I’ll dig into the logs _(which go back 15 days, ugh)_. I’ll also add `wsrep_sync_wait = 1` to the creation script to prevent this from happening again. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [October 29, 2025, 1:47pm UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/7 "2025-10-29T13:47:16Z")
`SELECT COUNT(*) FROM mysql.user` if that is over 1000 entries, then that is probably the issue. There is an internal caching table that holds user permissions for fast lookup. If you have that high of users, then the delay would be understandable. Each time you CREATE USER, or FLUSH PRIVILEGES, that table is destroyed and rebuilt. ---
**Author:** ![TAndreotti](https://avatars.discourse-cdn.com/v4/letter/t/8baadc/32.png) [@TAndreotti](https://forums.percona.com/u/TAndreotti)\ **Post date:** [October 29, 2025, 3:16pm UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/8 "2025-10-29T15:16:57Z")
1067 users. 1047 database. So, look like i’ll add a simple sleep in my script 😕 ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [October 29, 2025, 5:05pm UTC](https://forums.percona.com/t/delay-in-user-synchronization-across-percona-xtradb-cluster-nodes/39626/9 "2025-10-29T17:05:46Z")
Don’t add a simple sleep, use `wsrep_sync_wait`. You can do that like this: ```auto mysql> SELECT /*+ SET_VAR(wsrep_sync_wait=1) */ user, host FROM mysql.user WHERE user = 'foobar'; ``` or do `SET SESSION wsrep_sync_wait=1`, then execute the query.