Adding PXC Cluster node that's behind a NAT

Hello - as title states, I’m having issues joining a new node to an existing 4-node cluster.

The new node is physically separated and behind a static NAT. I can address the existing nodes from the new node and get the mysql> prompt from the command line. When attempting to start PXC on the new node, it gets as far as selecting a donor that I define, and placing said donor into state “Donor/Desynced”, as expected. After that, the new node errors out with a “Broken pipe” error, as seen here:

2021-07-28 17:17:34 13902 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role ‘joiner’ --address ‘xx.xxx.xxx.89’ --datadir ‘/var/lib/mysql/’ --defaults-file ‘/etc/my.cnf’ --defaults-group-suffix ‘’ --parent ‘13902’ --mysqld-version ‘5.6.51-91.0-56’ ‘’ : 32 (Broken pipe)
2021-07-28 17:17:34 13902 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2021-07-28 17:17:34 13902 [ERROR] WSREP: SST script aborted with error 32 (Broken pipe)
2021-07-28 17:17:34 13902 [ERROR] WSREP: SST failed: 32 (Broken pipe)
2021-07-28 17:17:34 13902 [ERROR] Aborting

The NAT translates the following to address our existing nodes (z nodes):

xx.xxx.xxx.89 => zz.zz.zz.49
xx.xxx.xxx.87 => zz.zz.zz.47
xx.xxx.xxx.83 => zz.zz.zz.43
xx.xxx.xxx.81 => zz.zz.zz.41

WSREP my.cnf options from new node here:
wsrep_cluster_address = gcomm://xx.xxx.xxx.89,xx.xxx.xxx.87,xx.xxx.xxx.83,xx.xxx.xxx.81
wsrep_provider = /usr/lib64/galera3/libgalera_smm.so
wsrep_slave_threads = 16
wsrep_sst_method = xtrabackup-v2
wsrep_sst_donor = [node name of donor]
wsrep_sst_auth = [user]:[pass]
wsrep_sst_receive_address = xx.xxx.xxx.89
wsrep_node_address = yy.yyy.yy.133
wsrep_cluster_name = my_wsrep_cluster
wsrep_node_name = Node3
wsrep_provider_options = “gcache.size = 5G”

Any ideas on this issue?

Thanks!

Because of the NAT, you’ll probably have to do some port mapping. You need to set incoming 4444, 4567, and 4568 to route to the inside-node.

Thank you - we are going to play around with port mapping to see if we can get the node to join