# One of the nodes periodically reconnects to another one because of timeouts

**URL:** <https://forums.percona.com/t/one-of-the-nodes-periodically-reconnects-to-another-one-because-of-timeouts/6948>\
**Category:** Percona XtraDB Cluster 5.x\
**Created:** [April 4, 2019, 7:42am UTC](https://forums.percona.com/t/one-of-the-nodes-periodically-reconnects-to-another-one-because-of-timeouts/6948 "2019-04-04T07:42:21Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![seri52](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@seri52](https://forums.percona.com/u/seri52)\
**Post date:** [April 4, 2019, 7:42am UTC](https://forums.percona.com/t/one-of-the-nodes-periodically-reconnects-to-another-one-because-of-timeouts/6948/1 "2019-04-04T07:42:21Z")

</div>

Hello!  
My cluster contains from 3 nodes:  
Amsterdam  
New York  
Frankfurt

OS: Ubuntu 16.04.6 LTS  
Percona Cluster version - 5.7  
my.cnf file:  
[mysqld]  
datadir=/var/lib/mysql  
user=mysql

# 

# The Percona XtraDB Cluster 5.7 configuration file.

# 

# 

# \* IMPORTANT: Additional settings that can override those from this file!

# The files must end with ‘.cnf’, otherwise they’ll be ignored.

# Please make any edits and changes to the appropriate sectional files

# included below.

# 

!includedir /etc/mysql/conf.d/  
!includedir /etc/mysql/percona-xtradb-cluster.conf.d/

wsrep\_provider=/usr/lib/libgalera\_smm.so  
max\_connections=400  
wsrep\_cluster\_name=pxc-cluster  
wsrep\_cluster\_address=gcomm://[nodes\_ips]  
wsrep\_node\_name=pxcny  
wsrep\_node\_address=[current\_node\_ip]

wsrep\_sst\_method=xtrabackup-v2  
wsrep\_sst\_auth=“sstuser:sstpass”

pxc\_strict\_mode=ENFORCING

binlog\_format=ROW  
default\_storage\_engine=InnoDB  
innodb\_autoinc\_lock\_mode=2

So! Sometimes 1-2 times a day NY node has those errors:  
2019-04-03T23:46:44.853439Z 708036 [Warning] IP address ‘125.64.94.200’ could not be resolved: Name or service not known  
2019-04-03T23:47:36.973584Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://amsterdam\_ip:4567  
2019-04-03T23:47:37.973708Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) reconnecting to 1f53f18d (tcp://amsterdam\_ip:4567), attempt 0  
2019-04-03T23:47:38.515111Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection established to 1f53f18d tcp://amsterdam\_ip:4567  
2019-04-03T23:47:39.467642Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection established to 1f53f18d tcp://amsterdam\_ip:4567  
2019-04-03T23:47:42.475200Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting off  
2019-04-03T23:47:45.975950Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://amsterdam\_ip:4567  
2019-04-03T23:47:46.973873Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection established to 1f53f18d tcp://amsterdam\_ip:4567  
2019-04-03T23:47:49.976568Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting off  
2019-04-04T03:20:42.184906Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection to peer 1f53f18d with addr tcp://amsterdam\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:42.185576Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://amsterdam\_ip:4567  
2019-04-04T03:20:43.185823Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) reconnecting to 1f53f18d (tcp://amsterdam\_ip:4567), attempt 0  
2019-04-04T03:20:46.195921Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection to peer 00000000 with addr tcp://amsterdam\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:47.611246Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection established to 1f53f18d tcp://amsterdam\_ip:4567  
2019-04-04T03:20:50.197061Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection to peer 00000000 with addr tcp://amsterdam\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:50.697396Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting off  
2019-04-04T03:20:52.197832Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://amsterdam\_ip:4567  
2019-04-04T03:20:53.198148Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) reconnecting to 1f53f18d (tcp://amsterdam\_ip:4567), attempt 0  
2019-04-04T03:20:55.607430Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection established to 1f53f18d tcp://amsterdam\_ip:4567  
2019-04-04T03:20:56.199301Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) connection to peer 00000000 with addr tcp://amsterdam\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:58.700023Z 0 [Note] WSREP: (252143b3, ‘tcp://0.0.0.0:4567’) turning message relay requesting off

Meanwhile on Amsterdam node:

2019-04-03T23:47:36.572541Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection to peer 252143b3 with addr tcp://ny\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-03T23:47:36.573546Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://ny\_ip:4567  
2019-04-03T23:47:38.072733Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) reconnecting to 252143b3 (tcp://ny\_ip:4567), attempt 0  
2019-04-03T23:47:38.475317Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection established to 252143b3 tcp://ny\_ip:4567  
2019-04-03T23:47:39.607324Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection established to 252143b3 tcp://ny\_ip:4567  
2019-04-03T23:47:43.073597Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) turning message relay requesting off  
2019-04-03T23:47:45.574354Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection to peer 252143b3 with addr tcp://ny\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-03T23:47:45.574533Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://ny\_ip:4567  
2019-04-03T23:47:46.574474Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) reconnecting to 252143b3 (tcp://ny\_ip:4567), attempt 0  
2019-04-03T23:47:47.583490Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection established to 252143b3 tcp://ny\_ip:4567  
2019-04-03T23:47:51.075158Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) turning message relay requesting off  
2019-04-04T03:20:40.711399Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection to peer 252143b3 with addr tcp://ny\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:40.711601Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers: tcp://ny\_ip:4567  
2019-04-04T03:20:42.211587Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) reconnecting to 252143b3 (tcp://ny\_ip:4567), attempt 0  
2019-04-04T03:20:45.712143Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection to peer 00000000 with addr tcp://ny\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:48.295260Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection established to 252143b3 tcp://ny\_ip:4567  
2019-04-04T03:20:51.712769Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection to peer 252143b3 with addr tcp://ny\_ip:4567 timed out, no messages seen in PT3S (gmcast.peer\_timeout)  
2019-04-04T03:20:53.212864Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) reconnecting to 252143b3 (tcp://ny\_ip:4567), attempt 0  
2019-04-04T03:20:55.746721Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) connection established to 252143b3 tcp://ny\_ip:4567  
2019-04-04T03:20:59.213535Z 0 [Note] WSREP: (1f53f18d, ‘tcp://0.0.0.0:4567’) turning message relay requesting off

This problem causes some timeout disconnects on our production game server.

I couldn’t find any information about this issue.  
Would be much appreciated for any help. Thanks!
