Created a new node and attempted to add it to the cluster, replacing an existing node. Do I need to bootstrap the main node again? I shut down all nodes and updated their my.cnf to reflect new node IP. Then attempted a restart.
wsrep_cluster_address=gcomm://45.56.115.5,45.33.103.216,50.116.45.43
From the error log:
2021-06-23T15:19:33.042694Z 0 [ERROR] [MY-000000] [Galera] failed to open gcomm backend connection: 110: failed to reach primary view (pc.wait_prim_timeout): 110 (Connection timed out)
at gcomm/src/pc.cpp:connect():161
2021-06-23T15:19:33.042734Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs_core.cpp:gcs_core_open():220: Failed to open backend connection: -110 (Connection timed out)
2021-06-23T15:19:34.043055Z 0 [Note] [MY-000000] [Galera] gcomm: terminating thread
2021-06-23T15:19:34.043137Z 0 [Note] [MY-000000] [Galera] gcomm: joining thread
2021-06-23T15:19:34.043396Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs.cpp:gcs_open():1758: Failed to open channel ‘iad-cluster’ at ‘gcomm://45.56.115.5,45.33.103.216,50.116.45.43’: -110 (Connection timed out)
2021-06-23T15:19:34.043429Z 0 [ERROR] [MY-000000] [Galera] gcs connect failed: Connection timed out
2021-06-23T15:19:34.043455Z 0 [ERROR] [MY-000000] [WSREP] Provider/Node (gcomm://45.56.115.5,45.33.103.216,50.116.45.43) failed to establish connection with cluster (reason: 7)