Node stuck with long semaphore wait in trx0undo.ic trx0purge.cc

MySQL & MariaDB Percona Server for MySQL 8.0
1

Hi all,

I’m running Percona XtraDB Cluster 8.0.23, and I’ve recently encountered multiple long semaphore waits in InnoDB, including: RW-latches in trx0undo.ic, buf0flu.cc, and trx0purge.cc.
This happened a few months ago, and it happened again today. It’s a cluster with three nodes and there’s not much load.
Any idea what could be wrong?

OS WAIT ARRAY INFO: reservation count 3188002
–Thread 140347480385280 has waited at trx0undo.ic line 143 for 518 seconds the semaphore:
S-lock on RW-latch at 0x7fa3c6c04a00 created in file buf0buf.cc line 789
a writer (thread id 140347479975680) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 143
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/include/trx0undo.ic line 129
–Thread 140347480180480 has waited at ha_innodb.cc line 3040 for 511 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
–Thread 140347479156480 has waited at ha_innodb.cc line 3040 for 508 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
–Thread 140341255669504 has waited at srv0srv.cc line 3620 for 517 seconds the semaphore:
X-lock (wait_ex) on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
–Thread 140346025563904 has waited at buf0flu.cc line 1352 for 568 seconds the semaphore:
SX-lock on RW-latch at 0x7fa3c6c04a00 created in file buf0buf.cc line 789
a writer (thread id 140347479975680) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 143
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/include/trx0undo.ic line 129
–Thread 140347482048256 has waited at ha_innodb.cc line 3040 for 516 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
–Thread 140347478337280 has waited at ha_innodb.cc line 3040 for 479 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439

Conf:
mysql Ver 8.0.23-14.1 for Linux on x86_64 (Percona XtraDB Cluster (GPL), Release rel14, Revision d3b9a1d, WSREP version 26.4.3)

innodb_buffer_pool_size=4G
innodb_buffer_pool_chunk_size = 1G
innodb_buffer_pool_instances=4
log_slave_updates = ON
binlog_expire_logs_seconds = 86400
key_buffer_size = 8M
max_allowed_packet = 512M
thread_stack = 192K
binlog_format=ROW
binlog_row_event_max_size=2M
binlog_cache_size=2M
wsrep_provider=/usr/lib/libgalera_smm.so
wsrep_provider_options="gcache.size=2G; gmcast.segment=0; gcs.max_packet_size=1048576; evs.send_window=512; evs.user_send_window=256; gcs.fc_limit=128; evs.inactive_timeout = PT90S; evs.suspect_timeout=PT30S"
wsrep_slave_threads= 16
default_storage_engine=InnoDB
wsrep_replicate_myisam=0
wsrep_log_conflicts
pxc_strict_mode=PERMISSIVE
innodb_purge_threads = 4
innodb_max_purge_lag = 0
innodb_max_purge_lag_delay = 0
innodb_undo_tablespaces = 2

I know I should upgrade my MySQL version, but could the problem be due to a configuration error? Is there a parameter of interest that’s missing?
I’d appreciate any help.
Thanks.

2

@JuanMag Looking over the semaphore outputs it seems some internal contention happening.

Mostly the threads seems waiting here.

   4 ha_innodb.cc 3040
   1 trx0undo.ic 143
   1 srv0srv.cc 3620
   1 buf0flu.cc 1352

Below are the os thread id holding the mutex.

   5 a writer (thread id 140341255669504) has reserved it in mode wait exclusive
   2 a writer (thread id 140347479975680) has reserved it in mode exclusive

Do you have the full output of show engine innodb status\G ? That will give some better picture. Please verify if these id’s associated with any DML or not ?

Yes, you should probably think about upgrading to much latest PXC version. You are using pretty older one and have greater chances of any past bug or problems.

I see one in 8.0.26 - Jira where the cluster locks around trx0undo.ic.

Well you can also check such waiting/locking DML with the help of below queries.

SELECT
 r.trx_id waiting_trx_id,
 r.trx_mysql_thread_id waiting_thread,
 r.trx_query waiting_query,
 b.trx_id blocking_trx_id,
 b.trx_mysql_thread_id blocking_thread,
 b.trx_query blocking_query
 FROM information_schema.innodb_lock_waits w
 INNER JOIN information_schema.innodb_trx b
 ON b.trx_id = w.blocking_trx_id
 INNER JOIN information_schema.innodb_trx r
 ON r.trx_id = w.requesting_trx_id;

OR

SELECT
 waiting_trx_id,
 waiting_pid,
 waiting_query,
 blocking_trx_id,
 blocking_pid,
 blocking_query
 FROM sys.innodb_lock_waits;

Find the exact locking query details with the below PS tables (if enabled).

mysql> SELECT THREAD_ID FROM performance_schema.threads WHERE PROCESSLIST_ID = xxx; **# from above**
mysql> SELECT THREAD_ID, SQL_TEXT FROM performance_schema.events_statements_current
 WHERE THREAD_ID =  xxx; **#from above**

Is this the writer node ? Other PXC/Galera nodes looks fine ?

2 Likes
3

Thank you very much for your response @anil.joshi
This only occurs on the writing node; the other nodes remain fine.

Here’s the full status.

-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 2085181 srv_active, 0 srv_shutdown, 14916193 srv_idle
srv_master_thread log flush and writes: 0
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 3187939
--Thread 140347480385280 has waited at trx0undo.ic line 143 for 238 seconds the semaphore:
S-lock on RW-latch at 0x7fa3c6c04a00 created in file buf0buf.cc line 789
a writer (thread id 140347479975680) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 143
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/include/trx0undo.ic line 129
--Thread 140347480180480 has waited at ha_innodb.cc line 3040 for 231 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347479156480 has waited at ha_innodb.cc line 3040 for 228 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140341255669504 has waited at srv0srv.cc line 3620 for 237 seconds the semaphore:
X-lock (wait_ex) on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346025563904 has waited at buf0flu.cc line 1352 for 288 seconds the semaphore:
SX-lock on RW-latch at 0x7fa3c6c04a00 created in file buf0buf.cc line 789
a writer (thread id 140347479975680) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 143
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/include/trx0undo.ic line 129
--Thread 140347482048256 has waited at ha_innodb.cc line 3040 for 236 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478337280 has waited at ha_innodb.cc line 3040 for 199 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476289280 has waited at ha_innodb.cc line 3040 for 199 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482253056 has waited at ha_innodb.cc line 3040 for 185 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347483301632 has waited at ha_innodb.cc line 3040 for 180 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476494080 has waited at ha_innodb.cc line 3040 for 161 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482482432 has waited at ha_innodb.cc line 3040 for 143 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478951680 has waited at ha_innodb.cc line 3040 for 122 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347479361280 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481433856 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481843456 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477313280 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347483506432 has waited at ha_innodb.cc line 3040 for 104 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347479566080 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481638656 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477927680 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482687232 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478746880 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347483096832 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347480794880 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481229056 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477518080 has waited at ha_innodb.cc line 3040 for 98 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482892032 has waited at ha_innodb.cc line 3040 for 94 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477108480 has waited at ha_innodb.cc line 3040 for 86 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476698880 has waited at ha_innodb.cc line 3040 for 81 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476903680 has waited at ha_innodb.cc line 3040 for 81 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478132480 has waited at ha_innodb.cc line 3040 for 80 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347480999680 has waited at ha_innodb.cc line 3040 for 77 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476084480 has waited at ha_innodb.cc line 3040 for 74 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347475879680 has waited at ha_innodb.cc line 3040 for 70 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347475674880 has waited at ha_innodb.cc line 3040 for 61 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347475470080 has waited at ha_innodb.cc line 3040 for 60 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
.......................

............................
--Thread 140346837010176 has waited at ha_innodb.cc line 3040 for 8 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
OS WAIT ARRAY INFO: signal count 3845522
RW-shared spins 997824, rounds 1003394, OS waits 5097
RW-excl spins 538729, rounds 973272, OS waits 21315
RW-sx spins 548, rounds 14346, OS waits 362
Spin rounds per wait: 1.01 RW-shared, 1.81 RW-excl, 26.18 RW-sx
------------------------
LATEST FOREIGN KEY ERROR
------------------------
2025-02-06 08:32:41
------------------------
LATEST DETECTED DEADLOCK
------------------------
2024-11-28 14:24:55 0x7fa3c2317700
------------
TRANSACTIONS
------------
Trx id counter 38460727
Purge done for trx's n:o < 38460722 undo n:o < 0 state: running but idle
History list length 4
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421821954392624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421821954389864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 38460726, ACTIVE (PREPARED) 261 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 6354427, OS thread handle 140347477722880, query id 477245987 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846841)
INSERT INTO table1 (`col1`, `col2`) VALUES ('val1', 'val2')
---TRANSACTION 38460725, ACTIVE (PREPARED) 264 sec
mysql tables in use 1, locked 1
1 lock struct(s), heap size 1136, 0 row lock(s), undo log entries 1
MySQL thread id 6354426, OS thread handle 140347480590080, query id 477245967 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846840)
INSERT INTO table2 (`col1`) VALUES ('val1')
---TRANSACTION 38460724, ACTIVE (PREPARED) 283 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 6354418, OS thread handle 140347478542080, query id 477245835 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846839)
UPDATE  table1 SET `col1` = 'val1'  WHERE col2= "val2"
---TRANSACTION 38460723, ACTIVE (PREPARED) 291 sec
mysql tables in use 1, locked 1
1 lock struct(s), heap size 1136, 0 row lock(s), undo log entries 1
MySQL thread id 6354415, OS thread handle 140347479770880, query id 477245764 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846838)
INSERT INTO table2 (`col1`) VALUES ('value1)
---TRANSACTION 38460720, ACTIVE (PREPARED) 291 sec committing
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 6354414, OS thread handle 140347479975680, query id 477245761 10.42.1.118 BD1 innobase_commit_low (3846836)
UPDATE  table3 SET `*************"
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
 ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 789
16235 OS file reads, 38023221 OS file writes, 22613743 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 2729, seg size 2731, 64 merges
merged operations:
 insert 61, delete mark 3, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 1106407, node heap has 15 buffer(s)
Hash table size 1106407, node heap has 135 buffer(s)
Hash table size 1106407, node heap has 7 buffer(s)
Hash table size 1106407, node heap has 11 buffer(s)
Hash table size 1106407, node heap has 10 buffer(s)
Hash table size 1106407, node heap has 1106 buffer(s)
Hash table size 1106407, node heap has 18 buffer(s)
Hash table size 1106407, node heap has 26 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number          201100517715
Log buffer assigned up to    201100517715
Log buffer completed up to   201100517715
Log written up to            201100517715
Log flushed up to            201100517715
Added dirty pages up to      201100517715
Pages flushed up to          201100495543
Last checkpoint at           201100495543
Checkpoint age target        177560064
Modified age no less than    22172
Checkpoint age               22172
Max checkpoint age           172491264
Number of logs               2
Log size                     104857600
Log total size               209715200
12684261 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 4383047680
Dictionary memory allocated 1007749
Buffer pool size   262140
Buffer pool size, bytes 4294901760
Free buffers       224957
Database pages     35855
Old database pages 13152
Modified db pages  9
Pending reads      0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 112834829, not young 186447
0.00 youngs/s, 0.00 non-youngs/s
Pages read 15943, created 21277, written 16859770
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 35855, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56408
Database pages     8804
Old database pages 3229
Modified db pages  0
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 31313985, not young 6151
0.00 youngs/s, 0.00 non-youngs/s
Pages read 3869, created 5275, written 4092235
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 8804, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56340
Database pages     8866
Old database pages 3252
Modified db pages  1
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 29447999, not young 55933
0.00 youngs/s, 0.00 non-youngs/s
Pages read 3951, created 5291, written 3259147
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 8866, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56108
Database pages     9094
Old database pages 3336
Modified db pages  5
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 28800444, not young 122011
0.00 youngs/s, 0.00 non-youngs/s
Pages read 4095, created 5318, written 4529187
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 9094, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56101
Database pages     9091
Old database pages 3335
Modified db pages  3
Pending reads      0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 23272401, not young 2352
0.00 youngs/s, 0.00 non-youngs/s
Pages read 4028, created 5393, written 4979201
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 9091, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
5 RW transactions active inside InnoDB
---OLDEST VIEW---
Read view low limit trx n:o 38460722
Trx read view will not see trx with id >= 38460727, sees < 38460720
Read view individually stored trx ids:
Read view trx id 38460720
Read view trx id 38460723
Read view trx id 38460724
Read view trx id 38460725
Read view trx id 38460726
-----------------
Process ID=3719, Main thread ID=140341289240320 , state=sleeping
Number of rows inserted 1233866, updated 2995686, deleted 831, read 258965807885
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
Number of system rows inserted 1293, updated 5905, deleted 1204, read 2476207
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------```


Thank you very much for your time
4

Here full semaphores output:

----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 3187939
--Thread 140347480385280 has waited at trx0undo.ic line 143 for 238 seconds the semaphore:
S-lock on RW-latch at 0x7fa3c6c04a00 created in file buf0buf.cc line 789
a writer (thread id 140347479975680) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 143
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/include/trx0undo.ic line 129
--Thread 140347480180480 has waited at ha_innodb.cc line 3040 for 231 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347479156480 has waited at ha_innodb.cc line 3040 for 228 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140341255669504 has waited at srv0srv.cc line 3620 for 237 seconds the semaphore:
X-lock (wait_ex) on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346025563904 has waited at buf0flu.cc line 1352 for 288 seconds the semaphore:
SX-lock on RW-latch at 0x7fa3c6c04a00 created in file buf0buf.cc line 789
a writer (thread id 140347479975680) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 143
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/include/trx0undo.ic line 129
--Thread 140347482048256 has waited at ha_innodb.cc line 3040 for 236 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478337280 has waited at ha_innodb.cc line 3040 for 199 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476289280 has waited at ha_innodb.cc line 3040 for 199 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482253056 has waited at ha_innodb.cc line 3040 for 185 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347483301632 has waited at ha_innodb.cc line 3040 for 180 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476494080 has waited at ha_innodb.cc line 3040 for 161 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482482432 has waited at ha_innodb.cc line 3040 for 143 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478951680 has waited at ha_innodb.cc line 3040 for 122 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347479361280 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481433856 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481843456 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477313280 has waited at ha_innodb.cc line 3040 for 111 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347483506432 has waited at ha_innodb.cc line 3040 for 104 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347479566080 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481638656 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477927680 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482687232 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478746880 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347483096832 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347480794880 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347481229056 has waited at ha_innodb.cc line 3040 for 103 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477518080 has waited at ha_innodb.cc line 3040 for 98 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347482892032 has waited at ha_innodb.cc line 3040 for 94 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347477108480 has waited at ha_innodb.cc line 3040 for 86 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476698880 has waited at ha_innodb.cc line 3040 for 81 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476903680 has waited at ha_innodb.cc line 3040 for 81 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347478132480 has waited at ha_innodb.cc line 3040 for 80 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347480999680 has waited at ha_innodb.cc line 3040 for 77 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347476084480 has waited at ha_innodb.cc line 3040 for 74 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347475879680 has waited at ha_innodb.cc line 3040 for 70 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347475674880 has waited at ha_innodb.cc line 3040 for 61 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140347475470080 has waited at ha_innodb.cc line 3040 for 60 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346839262976 has waited at ha_innodb.cc line 3040 for 52 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346839058176 has waited at ha_innodb.cc line 3040 for 44 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346838853376 has waited at ha_innodb.cc line 3040 for 43 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346838648576 has waited at ha_innodb.cc line 3040 for 43 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346838443776 has waited at ha_innodb.cc line 3040 for 42 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346838238976 has waited at ha_innodb.cc line 3040 for 38 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346838034176 has waited at ha_innodb.cc line 3040 for 33 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346837829376 has waited at ha_innodb.cc line 3040 for 22 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346837624576 has waited at ha_innodb.cc line 3040 for 14 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346837419776 has waited at ha_innodb.cc line 3040 for 13 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346837214976 has waited at ha_innodb.cc line 3040 for 11 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
--Thread 140346837010176 has waited at ha_innodb.cc line 3040 for 8 seconds the semaphore:
S-lock on RW-latch at 0x7fa5183911b0 created in file trx0purge.cc line 232
a writer (thread id 140341255669504) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1, lock_word: ffffffffffffffff
Last time read locked in file trx0rec.cc line 2439
Last time write locked in file /mnt/jenkins/workspace/pxc80-autobuild-RELEASE/test/percona-xtradb-cluster-8.0.23-14/storage/innobase/srv/srv0srv.cc line 3583
OS WAIT ARRAY INFO: signal count 3845522
RW-shared spins 997824, rounds 1003394, OS waits 5097
RW-excl spins 538729, rounds 973272, OS waits 21315
RW-sx spins 548, rounds 14346, OS waits 362
Spin rounds per wait: 1.01 RW-shared, 1.81 RW-excl, 26.18 RW-sx
5

Thank you very much for your time @anil.joshi

I think my previous reply has been blocked…
This only happens on the write node.
Here’s the output.

-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 2085181 srv_active, 0 srv_shutdown, 14916193 srv_idle
srv_master_thread log flush and writes: 0

------------------------
LATEST FOREIGN KEY ERROR
------------------------
2025-02-06 08:32:41
------------------------
LATEST DETECTED DEADLOCK
------------------------
2024-11-28 14:24:55 0x7fa3c2317700
------------
TRANSACTIONS
------------
Trx id counter 38460727
Purge done for trx's n:o < 38460722 undo n:o < 0 state: running but idle
History list length 4
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421821954392624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421821954389864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 38460726, ACTIVE (PREPARED) 261 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 6354427, OS thread handle 140347477722880, query id 477245987 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846841)
INSERT INTO table1 (`col1`, `col2`) VALUES ('val1', 'val2')
---TRANSACTION 38460725, ACTIVE (PREPARED) 264 sec
mysql tables in use 1, locked 1
1 lock struct(s), heap size 1136, 0 row lock(s), undo log entries 1
MySQL thread id 6354426, OS thread handle 140347480590080, query id 477245967 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846840)
INSERT INTO table2 (`col1`) VALUES ('val1')
---TRANSACTION 38460724, ACTIVE (PREPARED) 283 sec
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 6354418, OS thread handle 140347478542080, query id 477245835 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846839)
UPDATE  table1 SET `col1` = 'val1'  WHERE col2= "val2"
---TRANSACTION 38460723, ACTIVE (PREPARED) 291 sec
mysql tables in use 1, locked 1
1 lock struct(s), heap size 1136, 0 row lock(s), undo log entries 1
MySQL thread id 6354415, OS thread handle 140347479770880, query id 477245764 10.42.1.118 BD1 wsrep: write set replicated and table0ified (3846838)
INSERT INTO table2 (`col1`) VALUES ('value1)
---TRANSACTION 38460720, ACTIVE (PREPARED) 291 sec committing
mysql tables in use 1, locked 1
2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
MySQL thread id 6354414, OS thread handle 140347479975680, query id 477245761 10.42.1.118 BD1 innobase_commit_low (3846836)
UPDATE  table3 SET `*************"
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
 ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 789
16235 OS file reads, 38023221 OS file writes, 22613743 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 2729, seg size 2731, 64 merges
merged operations:
 insert 61, delete mark 3, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 1106407, node heap has 15 buffer(s)
Hash table size 1106407, node heap has 135 buffer(s)
Hash table size 1106407, node heap has 7 buffer(s)
Hash table size 1106407, node heap has 11 buffer(s)
Hash table size 1106407, node heap has 10 buffer(s)
Hash table size 1106407, node heap has 1106 buffer(s)
Hash table size 1106407, node heap has 18 buffer(s)
Hash table size 1106407, node heap has 26 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number          201100517715
Log buffer assigned up to    201100517715
Log buffer completed up to   201100517715
Log written up to            201100517715
Log flushed up to            201100517715
Added dirty pages up to      201100517715
Pages flushed up to          201100495543
Last checkpoint at           201100495543
Checkpoint age target        177560064
Modified age no less than    22172
Checkpoint age               22172
Max checkpoint age           172491264
Number of logs               2
Log size                     104857600
Log total size               209715200
12684261 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 4383047680
Dictionary memory allocated 1007749
Buffer pool size   262140
Buffer pool size, bytes 4294901760
Free buffers       224957
Database pages     35855
Old database pages 13152
Modified db pages  9
Pending reads      0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 112834829, not young 186447
0.00 youngs/s, 0.00 non-youngs/s
Pages read 15943, created 21277, written 16859770
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 35855, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56408
Database pages     8804
Old database pages 3229
Modified db pages  0
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 31313985, not young 6151
0.00 youngs/s, 0.00 non-youngs/s
Pages read 3869, created 5275, written 4092235
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 8804, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56340
Database pages     8866
Old database pages 3252
Modified db pages  1
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 29447999, not young 55933
0.00 youngs/s, 0.00 non-youngs/s
Pages read 3951, created 5291, written 3259147
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 8866, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56108
Database pages     9094
Old database pages 3336
Modified db pages  5
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 28800444, not young 122011
0.00 youngs/s, 0.00 non-youngs/s
Pages read 4095, created 5318, written 4529187
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 9094, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size   65535
Buffer pool size, bytes 1073725440
Free buffers       56101
Database pages     9091
Old database pages 3335
Modified db pages  3
Pending reads      0
Pending writes: LRU 0, flush list 2, single page 0
Pages made young 23272401, not young 2352
0.00 youngs/s, 0.00 non-youngs/s
Pages read 4028, created 5393, written 4979201
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 9091, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
5 RW transactions active inside InnoDB
---OLDEST VIEW---
Read view low limit trx n:o 38460722
Trx read view will not see trx with id >= 38460727, sees < 38460720
Read view individually stored trx ids:
Read view trx id 38460720
Read view trx id 38460723
Read view trx id 38460724
Read view trx id 38460725
Read view trx id 38460726
-----------------
Process ID=3719, Main thread ID=140341289240320 , state=sleeping
Number of rows inserted 1233866, updated 2995686, deleted 831, read 258965807885
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
Number of system rows inserted 1293, updated 5905, deleted 1204, read 2476207
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------```