Hello,
We have a 3 node percona cluster, with first node bootstrapped, we are trying to join node 2 and 3 to the cluster, but mysql service is failing after it tries to attempt SST.
2025-02-19T08:44:39.527965-05:00 0 [ERROR] [MY-000000] [Galera] async IST sender failed to serve tcp://10.32.74.231:4568: ist send failed: ', asio error ‘Failed to read: Connection reset by peer: 104 (Connection reset by peer)
at galerautils/src/gu_asio_stream_react.cpp:throw_sync_op_error():141’: 104 (Connection reset by peer)
at galera/src/ist.cpp:send():862
2025-02-19T08:44:39.528075-05:00 0 [Note] [MY-000000] [Galera] async IST sender served
2025-02-19T08:44:39.671549-05:00 0 [Warning] [MY-000000] [WSREP-SST] wsrep_node_address or wsrep_sst_receive_address not set. Consider setting them if SST fails.
2025-02-19T08:44:39.887101-05:00 26515 [Warning] [MY-013712] [Server] No suitable ‘keyring_component_metadata_query’ service implementation found to fulfill the request.
2025-02-19T08:44:39.904310-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:39 socat[1610805] E write(7, 0x5614d46b5000, 168): Connection refused
2025-02-19T08:44:40.911086-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:40 socat[1610822] E write(7, 0x562461384000, 168): Connection refused
2025-02-19T08:44:41.917966-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:41 socat[1610825] E write(7, 0x56184b596000, 146): Connection refused
2025-02-19T08:44:42.924766-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:42 socat[1610828] E write(7, 0x55eb20845000, 168): Connection refused
2025-02-19T08:44:43.931327-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:43 socat[1610845] E write(7, 0x564595457000, 168): Connection refused
2025-02-19T08:44:44.531016-05:00 0 [Note] [MY-000000] [Galera] cleaning up 680836fa-8def (tcp://10.32.74.231:4567)
2025-02-19T08:44:44.937792-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:44 socat[1610848] E write(7, 0x5652f4b21000, 168): Connection refused
2025-02-19T08:44:45.944729-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:45 socat[1610851] E write(7, 0x564e9ff42000, 168): Connection refused
2025-02-19T08:44:46.951024-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:46 socat[1610868] E write(7, 0x55b211263000, 168): Connection refused
2025-02-19T08:44:47.958345-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:47 socat[1610871] E write(7, 0x5558906e3000, 168): Connection refused
2025-02-19T08:44:48.965889-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:48 socat[1610874] E write(7, 0x56348728d000, 168): Connection refused
2025-02-19T08:44:49.973197-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:49 socat[1610891] E write(7, 0x5613386a8000, 168): Connection refused
donor is showing below errors:-
2025-02-19T08:44:39.527965-05:00 0 [ERROR] [MY-000000] [Galera] async IST sender failed to serve tcp://10.32.74.231:4568: ist send failed: ', asio error ‘Failed to read: Connection reset by peer: 104 (Connection reset by peer)
at galerautils/src/gu_asio_stream_react.cpp:throw_sync_op_error():141’: 104 (Connection reset by peer)
at galera/src/ist.cpp:send():862
2025-02-19T08:44:39.528075-05:00 0 [Note] [MY-000000] [Galera] async IST sender served
2025-02-19T08:44:39.671549-05:00 0 [Warning] [MY-000000] [WSREP-SST] wsrep_node_address or wsrep_sst_receive_address not set. Consider setting them if SST fails.
2025-02-19T08:44:39.887101-05:00 26515 [Warning] [MY-013712] [Server] No suitable ‘keyring_component_metadata_query’ service implementation found to fulfill the request.
2025-02-19T08:44:39.904310-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:39 socat[1610805] E write(7, 0x5614d46b5000, 168): Connection refused
2025-02-19T08:44:40.911086-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:40 socat[1610822] E write(7, 0x562461384000, 168): Connection refused
2025-02-19T08:44:41.917966-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:41 socat[1610825] E write(7, 0x56184b596000, 146): Connection refused
2025-02-19T08:44:42.924766-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:42 socat[1610828] E write(7, 0x55eb20845000, 168): Connection refused
2025-02-19T08:44:43.931327-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:43 socat[1610845] E write(7, 0x564595457000, 168): Connection refused
2025-02-19T08:44:44.531016-05:00 0 [Note] [MY-000000] [Galera] cleaning up 680836fa-8def (tcp://10.32.74.231:4567)
2025-02-19T08:44:44.937792-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:44 socat[1610848] E write(7, 0x5652f4b21000, 168): Connection refused
2025-02-19T08:44:45.944729-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:45 socat[1610851] E write(7, 0x564e9ff42000, 168): Connection refused
2025-02-19T08:44:46.951024-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:46 socat[1610868] E write(7, 0x55b211263000, 168): Connection refused
2025-02-19T08:44:47.958345-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:47 socat[1610871] E write(7, 0x5558906e3000, 168): Connection refused
2025-02-19T08:44:48.965889-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:48 socat[1610874] E write(7, 0x56348728d000, 168): Connection refused
2025-02-19T08:44:49.973197-05:00 0 [Note] [MY-000000] [WSREP-SST] 2025/02/19 08:44:49 socat[1610891] E write(7, 0x5613386a8000, 168): Connection refused
There are no permission issues with /usr
, /usr/bin
, /usr/bin/ls
, /proc
or /usr/bin/wsrep_sst_xtrabackup-v2
. There is no connectivity issue between the servers on port 4444
, 4567
and port 4568
selinux and local firewall disabled, no iptables entries.
Can someone please help us fix this issue.
-Abhi