Error when adding node to cluster

MySQL & MariaDB Percona XtraDB Cluster 8.x
#1

Created a new node and attempted to add it to the cluster, replacing an existing node. Do I need to bootstrap the main node again? I shut down all nodes and updated their my.cnf to reflect new node IP. Then attempted a restart.

wsrep_cluster_address=gcomm://45.56.115.5,45.33.103.216,50.116.45.43

From the error log:

2021-06-23T15:19:33.042694Z 0 [ERROR] [MY-000000] [Galera] failed to open gcomm backend connection: 110: failed to reach primary view (pc.wait_prim_timeout): 110 (Connection timed out)
at gcomm/src/pc.cpp:connect():161
2021-06-23T15:19:33.042734Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs_core.cpp:gcs_core_open():220: Failed to open backend connection: -110 (Connection timed out)
2021-06-23T15:19:34.043055Z 0 [Note] [MY-000000] [Galera] gcomm: terminating thread
2021-06-23T15:19:34.043137Z 0 [Note] [MY-000000] [Galera] gcomm: joining thread
2021-06-23T15:19:34.043396Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs.cpp:gcs_open():1758: Failed to open channel ‘iad-cluster’ at ‘gcomm://45.56.115.5,45.33.103.216,50.116.45.43’: -110 (Connection timed out)
2021-06-23T15:19:34.043429Z 0 [ERROR] [MY-000000] [Galera] gcs connect failed: Connection timed out
2021-06-23T15:19:34.043455Z 0 [ERROR] [MY-000000] [WSREP] Provider/Node (gcomm://45.56.115.5,45.33.103.216,50.116.45.43) failed to establish connection with cluster (reason: 7)

1 Like
#2

Hello @Mikem,
Any time you completely stop the cluster (all nodes) you must always bootstrap the first node. This informs that node that it should not look to join an existing cluster.

1 Like
#3

Thanks Matthew, kind if what I figured, So to add a new node, do I need to update the wsrep_cluster_address in my.cnf on all running nodes including the new one and simply start the new node?

1 Like
#4

Technically, no. wsrep_cluster_address is not the definition of your cluster. The joining node only needs to know about 1 other node. As part of the communications protocol, nodes will share IPs of their connected nodes with all other connected nodes. Best practice says to add all nodes to each wsrep_cluster_address but it is not a requirement.

1 Like