Using garbd for hotbackup / semiautomatic slave setup

MySQL & MariaDB Percona XtraDB Cluster 5.x
1

Hi there,
I try to setup a process using the percona arbiter for fast setup of slave servers out of running percona xtradb clusters and have problems with using the garbd binary for this …
My problem is, that I can start the garbd and connect it to a cluster node, but the arbiter does not send the hostname or IP address to the cluster node and the there startet wsrep_sst_xtrabackup-v2 has no remote host where to stream to backup to …

I try it in this way:

garbd --address gcomm://db97.prv.fra.sprd.net:57192 --group sbo_test_cluster_fra --sst xtrabackup-v2

And on the cluster node I get this log messages:

2018-12-11T12:02:04.210448Z 0 [Note] WSREP: Service thread queue flushed.
2018-12-11T12:02:04.366836Z WSREP_SST: [WARNING] wsrep_node_address or wsrep_sst_receive_address not set. Consider setting them if SST fails.
2018-12-11T12:02:04.570683Z WSREP_SST: [ERROR] ******************* FATAL ERROR **********************
2018-12-11T12:02:04.571673Z WSREP_SST: [ERROR] Upgrade joiner to 5.6.21 or higher for backup locks support
2018-12-11T12:02:04.572605Z WSREP_SST: [ERROR] The joiner is not supported for this version of donor
2018-12-11T12:02:04.573521Z WSREP_SST: [ERROR] ******************************************************
2018-12-11T12:02:04.574700Z WSREP_SST: [ERROR] Cleanup after exit with status:93
2018-12-11T12:02:04.583151Z 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role ‘donor’ --address ‘’ --socket ‘/local/mysql/sbo_test_cluster/mys
ql.sock’ --datadir ‘/local/mysql/sbo_test_cluster/data/’ --defaults-file ‘/local/mysql/sbo_test_cluster/my.cnf’ --defaults-group-suffix ‘’ --mysqld-version ‘5.7.23-23-57’
–binlog ‘/local/mysql/sbo_test_cluster/log/master-bin’ --gtid ‘8168916f-fd2f-11e8-846a-07319d3aee35:0’ : 93 (Protocol not supported)
2018-12-11T12:02:04.583217Z 0 [ERROR] WSREP: Command did not run: wsrep_sst_xtrabackup-v2 --role ‘donor’ --address ‘’ --socket ‘/local/mysql/sbo_test_cluster/mysql.sock’
–datadir ‘/local/mysql/sbo_test_cluster/data/’ --defaults-file ‘/local/mysql/sbo_test_cluster/my.cnf’ --defaults-group-suffix ‘’ --mysqld-version ‘5.7.23-23-57’ --binlo
g ‘/local/mysql/sbo_test_cluster/log/master-bin’ --gtid ‘8168916f-fd2f-11e8-846a-07319d3aee35:0’

Is it somehow possible to provide the hostname of the remote host via parameter to the garbd process?
If not, how is the best way to start a wsrep_sst_xtrabackup on the remote host to start streaming a hotcopy of the running server to the server, where the backup shall be stored …?

Since a new cluster node uses the same way to get the data and join to the cluster, it must be possible somehow …
But I have no clue how … the joining node somehow starts the remote process while connecting to the running nodes mysql process and starts the process there magically …
I would like to do by hand without login in via ssh …

Hope somebody can give some hints and share ideas how to do …

Thanks
Steffen

2

Hi Steffen,

I think that using garbd for backups is an over-complicated hack rather then regular solution. Also, this method is not well tested and new PXC releases and SST scripts changes apparently made it more complicated.

With Xtrabackup and some additional tools, it is very easy to make a streaming backup, very similar to native SST process. You can find some examples here: [url]https://www.percona.com/doc/percona-xtrabackup/2.4/howtos/recipes_ibkx_stream.html[/url]
For PXC though, it is important to switch the node to desynced state before the backup, and use --galera-info option for xtrabackup.

Example of this method, very similar for PXC is presented here (just skip the group replication specific parts): [url]https://www.percona.com/blog/2018/11/05/how-to-quickly-add-a-node-to-an-innodb-cluster-or-group-replication/[/url]

I hope this works well for you.