# One of the nodes in the Galera Cluster is down with error "mysqld got signal 11"

**URL:** <https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341>\
**Category:** Percona XtraDB Cluster 8.x\
**Tags:** mysql\
**Created:** [September 12, 2025, 7:34am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341 "2025-09-12T07:34:23Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 12, 2025, 7:34am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/1 "2025-09-12T07:34:23Z")

</div>

here is the error

2025-09-11T09:00:50Z UTC - mysqld got signal 11 ;  
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.  
BuildID[sha1]=ae1efca2c0800a60ab995705a17c005525d455fc  
Thread pointer: 0x7fa07afb09d0  
Attempting backtrace. You can use the following information to find out  
where mysqld died. If you see no messages after this, something went  
terribly wrong…  
stack\_bottom = 7fa2e457ebd0 thread\_stack 0x100000  
/usr/local/mysql/bin/mysqld(my\_print\_stacktrace(unsigned char const\*, unsigned long)+0x3d) [0x21ccb8d]  
/usr/local/mysql/bin/mysqld(print\_fatal\_signal(int)+0x393) [0x1035bf3]  
/usr/local/mysql/bin/mysqld(handle\_fatal\_signal+0xa5) [0x1035ca5]  
/lib64/libpthread.so.0(+0x12990) [0x7fa451d9b990]  
/usr/local/mysql/bin/mysqld(wsrep\_handle\_mdl\_conflict(MDL\_context const\*, MDL\_ticket\*)+0x91) [0x10559c1]  
/usr/local/mysql/bin/mysqld(MDL\_lock::can\_grant\_lock(enum\_mdl\_type, MDL\_context const\*) const+0x4fb) [0x135c84b]  
/usr/local/mysql/bin/mysqld(MDL\_context::try\_acquire\_lock\_impl(MDL\_request\*, MDL\_ticket\*\*)+0x6ba) [0x136020a]  
/usr/local/mysql/bin/mysqld(MDL\_context::acquire\_lock(MDL\_request\*, unsigned long)+0xa6) [0x1360676]  
/usr/local/mysql/bin/mysqld(MDL\_context::acquire\_locks(I\_P\_List\<MDL\_request, I\_P\_List\_adapter\<MDL\_request, &MDL\_request::next\_in\_list, &MDL\_request::prev\_in\_list\>, I\_P\_List\_counter, I\_P\_List\_no\_push\_back\<MDL\_request\> \>_, unsigned long)+0x2b2) [0x13618d2]  
/usr/local/mysql/bin/mysqld() [0xf5c345]  
/usr/local/mysql/bin/mysqld(mysql\_alter\_table(THD_, char const\*, char const\*, HA\_CREATE\_INFO\*, Table\_ref\*, Alter\_info\*)+0x5326) [0xf7bd16]  
/usr/local/mysql/bin/mysqld(Sql\_cmd\_alter\_table::execute(THD\*)+0x6cb) [0x13f922b]  
/usr/local/mysql/bin/mysqld(mysql\_execute\_command(THD\*, bool)+0x1529) [0xec0379]  
/usr/local/mysql/bin/mysqld(dispatch\_sql\_command(THD\*, Parser\_state\*)+0x520) [0xec52d0]  
/usr/local/mysql/bin/mysqld() [0xec55bb]  
/usr/local/mysql/bin/mysqld(dispatch\_command(THD\*, COM\_DATA const\*, enum\_server\_command)+0x1b92) [0xec85a2]  
/usr/local/mysql/bin/mysqld(do\_command(THD\*)+0x2e3) [0xecb4c3]  
/usr/local/mysql/bin/mysqld() [0x1025038]  
/usr/local/mysql/bin/mysqld() [0x28873a5]  
/lib64/libpthread.so.0(+0x81ca) [0x7fa451d911ca]  
/lib64/libc.so.6(clone+0x43) [0x7fa4500fd8d3]

Trying to get some variables.  
Some pointers may be invalid and cause the dump to abort.  
Query (7fa0104727f0): is an invalid pointer  
Connection ID (thread ID): 1380845  
Status: NOT\_KILLED

version mysql 8.0.41

version galera 4.23

For the detailed error log, please refer to **error3306** ; for the system log, please refer to **messages** ; and for the configuration file, please refer to **my.txt**. All these files have been uploaded in the reply post I made below.

---

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 12, 2025, 7:37am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/2 "2025-09-12T07:37:04Z")

</div>

here is the error

2025-09-11T09:00:50Z UTC - mysqld got signal 11 ;  
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.  
BuildID[sha1]=ae1efca2c0800a60ab995705a17c005525d455fc  
Thread pointer: 0x7fa07afb09d0  
Attempting backtrace. You can use the following information to find out  
where mysqld died. If you see no messages after this, something went  
terribly wrong…  
stack\_bottom = 7fa2e457ebd0 thread\_stack 0x100000  
/usr/local/mysql/bin/mysqld(my\_print\_stacktrace(unsigned char const\*, unsigned long)+0x3d) [0x21ccb8d]  
/usr/local/mysql/bin/mysqld(print\_fatal\_signal(int)+0x393) [0x1035bf3]  
/usr/local/mysql/bin/mysqld(handle\_fatal\_signal+0xa5) [0x1035ca5]  
/lib64/libpthread.so.0(+0x12990) [0x7fa451d9b990]  
/usr/local/mysql/bin/mysqld(wsrep\_handle\_mdl\_conflict(MDL\_context const\*, MDL\_ticket\*)+0x91) [0x10559c1]  
/usr/local/mysql/bin/mysqld(MDL\_lock::can\_grant\_lock(enum\_mdl\_type, MDL\_context const\*) const+0x4fb) [0x135c84b]  
/usr/local/mysql/bin/mysqld(MDL\_context::try\_acquire\_lock\_impl(MDL\_request\*, MDL\_ticket\*\*)+0x6ba) [0x136020a]  
/usr/local/mysql/bin/mysqld(MDL\_context::acquire\_lock(MDL\_request\*, unsigned long)+0xa6) [0x1360676]  
/usr/local/mysql/bin/mysqld(MDL\_context::acquire\_locks(I\_P\_List\<MDL\_request, I\_P\_List\_adapter\<MDL\_request, &MDL\_request::next\_in\_list, &MDL\_request::prev\_in\_list\>, I\_P\_List\_counter, I\_P\_List\_no\_push\_back\<MDL\_request\> \>_, unsigned long)+0x2b2) [0x13618d2]  
/usr/local/mysql/bin/mysqld() [0xf5c345]  
/usr/local/mysql/bin/mysqld(mysql\_alter\_table(THD_, char const\*, char const\*, HA\_CREATE\_INFO\*, Table\_ref\*, Alter\_info\*)+0x5326) [0xf7bd16]  
/usr/local/mysql/bin/mysqld(Sql\_cmd\_alter\_table::execute(THD\*)+0x6cb) [0x13f922b]  
/usr/local/mysql/bin/mysqld(mysql\_execute\_command(THD\*, bool)+0x1529) [0xec0379]  
/usr/local/mysql/bin/mysqld(dispatch\_sql\_command(THD\*, Parser\_state\*)+0x520) [0xec52d0]  
/usr/local/mysql/bin/mysqld() [0xec55bb]  
/usr/local/mysql/bin/mysqld(dispatch\_command(THD\*, COM\_DATA const\*, enum\_server\_command)+0x1b92) [0xec85a2]  
/usr/local/mysql/bin/mysqld(do\_command(THD\*)+0x2e3) [0xecb4c3]  
/usr/local/mysql/bin/mysqld() [0x1025038]  
/usr/local/mysql/bin/mysqld() [0x28873a5]  
/lib64/libpthread.so.0(+0x81ca) [0x7fa451d911ca]  
/lib64/libc.so.6(clone+0x43) [0x7fa4500fd8d3]

Trying to get some variables.  
Some pointers may be invalid and cause the dump to abort.  
Query (7fa0104727f0): is an invalid pointer  
Connection ID (thread ID): 1380845  
Status: NOT\_KILLED

version mysql 8.0.41

version galera 4.23

```auto
For the detailed error log, please refer to error3306; for the system log, please refer to messages; and for the configuration file, please refer to my.txt.

```

[messages.txt](https://forums.percona.com/uploads/short-url/wnjTBpZSMFCRSilHX4WXtJFjUUL.txt) (808.7 KB)

[my.txt](https://forums.percona.com/uploads/short-url/1mIu84TnMSBcnCNMRtWnCLhgums.txt) (2.8 KB)

[error3306.log](https://forums.percona.com/uploads/short-url/jR0PyMd0RbSQLzPtsykMJgGAJa.log) (389.3 KB)

---

<div class="post-metadata">

**Author:** ![Wayne\_Leutwyler](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/wayne_leutwyler/32/20829_2.png) [@Wayne\_Leutwyler](https://forums.percona.com/u/Wayne_Leutwyler)\
**Post date:** [September 12, 2025, 11:49am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/3 "2025-09-12T11:49:39Z")

</div>

I see the following error in your log files:

```auto
rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(792) \[sender=3.1.3\]
WSREP_SST: \[ERROR\] find/rsync returned code 123: (20250908 13:46:26.663)
2025-09-08T13:46:26.665673+08:00 23 \[ERROR\] \[MY-000000\] \[WSREP\] Failed to read from: wsrep_sst_rsync --role 'donor' --address '172.21.100.23:4444/rsync_sst' --local-port '3306' --socket '/data/mes_mysql_log/3306.sock' --datadir '/data/mes_mysql_data/' --mysqld-version '8.0.41' --protocol '7' --plugin-dir '/usr/local/mysql/lib/plugin/' '' --binlog-index '3306-bin.index' --gtid 8320dd70-8c72-11f0-a0f9-139a0b29bc4d:11516 --local-gtid 8320dd70-8c72-11f0-a0f9-139a0b29bc4d:0 --server-id 1 --server-uuid 8146d93e-8c72-11f0-9518-f0d4e2e8a308

```

This issue appears to be related to **rsync** during state transfer. Have you verified end-to-end network connectivity between this node and the other cluster members (e.g., checking TCP reachability, firewall rules, and MTU consistency)?

Another error I see is:

```auto
2025-09-08T13:27:39.851025+08:00 15 [ERROR] [MY-013117] [Repl] Replica I/O for channel '': Fatal error: The replica I/O thread stops because source and replica have equal MySQL server ids; these ids must be different for replication to work (or the --replicate-same-server-id option must be used on replica but this does not always make sense; please check the manual before using it). Error_code: MY-013117

```

It appears that this node has an asynchronous replica attached, but both the primary node and the replica are configured with the same `server_id`. This configuration conflict will prevent replication from functioning correctly.

Additionally, the **Signal 11 (segmentation fault)** indicates that the MySQL process crashed unexpectedly. After the crash, did you attempt to restart the node? If so, was it able to rejoin the cluster and establish connectivity?

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [September 12, 2025, 2:06pm UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/4 "2025-09-12T14:06:06Z")

</div>

@ocean,  
Switch to using xtrabackupv2 as the SST method. The rsync method is extremely old, and may have issues with larger data sets. The xtrabackup method uses the latest features, and most recent versions to handle multi-TB datasets without issue.

---

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 15, 2025, 12:38am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/5 "2025-09-15T00:38:54Z")

</div>

Thank you very much for your reply. We have two Galera clusters, and the node with the issue belongs to a cluster that once served as the slave database of the other cluster. A switchover was performed on September 8th, and after the switchover, the cluster was used normally until September 11th.

After the outage, the node has not yet been restarted. This is because we have configured a full rsync synchronization, which requires a low-traffic period of business operations to complete the restart. Additionally, the root cause of the database outage has not been identified yet.

---

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 15, 2025, 12:42am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/6 "2025-09-15T00:42:12Z")

</div>

Thank you for taking the time to assist us. We plan to switch our full synchronization method from the current one to the xtrabackup-v2 method in the future.

The only difference between this faulty node and the other two normally functioning nodes is that we perform a daily logical backup on this node using mysqldump. I have a suspicion: could this logical backup be the cause of the database failure?

---

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 15, 2025, 1:04am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/7 "2025-09-15T01:04:18Z")

</div>

Additionally, I noticed that there were quite a few issues related to metadata locks when the failure occurred, so I also suspect whether the failure was caused by DDL operations. After checking the logs, I confirmed that there were indeed a large number of DDL operations. However, it is important to note that similar DDL statements have existed for a long time.

---

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 15, 2025, 9:40am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/8 "2025-09-15T09:40:16Z")

</div>

I still have one question. From the logs, we can see that it was still running at 17:00 on September 11th, but the time when “mysqld got signal 11” occurred was 09:00 in the morning.

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [September 15, 2025, 1:59pm UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/9 "2025-09-15T13:59:00Z")

</div>

> [@ocean](#):
>
> node using mysqldump. I have a suspicion: could this logical backup be the cause of the database failure?

Unlikely, however you should stop using mysqldump and switch to [mydumper](https://mydumper.github.io/mydumper/). Mydumper will complete your logical backup in 1/2 the amount of time, if not more, than mysqldump due to Mydumper being multi-threaded.

> [@ocean](#):
>
> I confirmed that there were indeed a large number of DDL operations

If you’re doing lots of DDL, _and_ attempting to take a backup at the same time, this could have been an issue.

---

<div class="post-metadata">

**Author:** ![ocean](https://avatars.discourse-cdn.com/v4/letter/o/13edae/32.png) [@ocean](https://forums.percona.com/u/ocean)\
**Post date:** [September 16, 2025, 12:55am UTC](https://forums.percona.com/t/one-of-the-nodes-in-the-galera-cluster-is-down-with-error-mysqld-got-signal-11/39341/10 "2025-09-16T00:55:48Z")

</div>

Thank you for your reply. Next, I will follow your suggestion: first stop the backup, then notify the R&D team to reduce DDL operations, observe for a period of time to see the effect, and then feed back to you. Thank you again for your help.