Adding PXC Cluster node that's behind a NAT

Hello - as title states, I’m having issues joining a new node to an existing 4-node cluster.

The new node is physically separated and behind a static NAT. I can address the existing nodes from the new node and get the mysql> prompt from the command line. When attempting to start PXC on the new node, it gets as far as selecting a donor that I define, and placing said donor into state “Donor/Desynced”, as expected. After that, the new node errors out with a “Broken pipe” error, as seen here:

2021-07-28 17:17:34 13902 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role ‘joiner’ --address ‘xx.xxx.xxx.89’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/my.cnf’ --defaults-group-suffix ‘’ --parent ‘13902’ --mysqld-version ‘5.6.51-91.0-56’ ‘’ : 32 (Broken pipe)
2021-07-28 17:17:34 13902 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2021-07-28 17:17:34 13902 [ERROR] WSREP: SST script aborted with error 32 (Broken pipe)
2021-07-28 17:17:34 13902 [ERROR] WSREP: SST failed: 32 (Broken pipe)
2021-07-28 17:17:34 13902 [ERROR] Aborting

The NAT translates the following to address our existing nodes (z nodes):

xx.xxx.xxx.89 => zz.zz.zz.49
xx.xxx.xxx.87 => zz.zz.zz.47
xx.xxx.xxx.83 => zz.zz.zz.43
xx.xxx.xxx.81 => zz.zz.zz.41

WSREP my.cnf options from new node here:
wsrep_cluster_address = gcomm://xx.xxx.xxx.89,xx.xxx.xxx.87,xx.xxx.xxx.83,xx.xxx.xxx.81
wsrep_provider = /usr/lib64/galera3/libgalera_smm.so
wsrep_slave_threads = 16
wsrep_sst_method = xtrabackup-v2
wsrep_sst_donor = [node name of donor]
wsrep_sst_auth = [user]:[pass]
wsrep_sst_receive_address = xx.xxx.xxx.89
wsrep_node_address = yy.yyy.yy.133
wsrep_cluster_name = my_wsrep_cluster
wsrep_node_name = Node3
wsrep_provider_options = “gcache.size = 5G”

Any ideas on this issue?

Thanks!

1 Like

Because of the NAT, you’ll probably have to do some port mapping. You need to set incoming 4444, 4567, and 4568 to route to the inside-node.

2 Likes

Thank you - we are going to play around with port mapping to see if we can get the node to join

1 Like