we have build up an five node percona-xtradb-cluster (version 8.0.22-13-1) on Ubuntu 20.04. Each Server has two dedicated 10GBit/s nics, 36 cores and 256 GB RAM and performs well when we do stuff like copy files etc.
Our config looks like:
[...] wsrep_provider_options="gcache.size=5G;gcache.page_size=1G;gmcast.listen_addr=tcp://10.10.20.11:4567;socket.ssl_key=server-key.pem;socket.ssl_cert=server-cert.pem;socket.ssl_ca=ca.pem" wsrep_slave_threads=18 wsrep_sst_method=xtrabackup-v2 performance_schema = True [sst] encrypt=4 ssl-key=server-key.pem ssl-ca=ca.pem ssl-cert=server-cert.pem inno-apply-opts="--use-memory=20G" encrypt_threads=16 [...]
When I add a new node, a full state transfer is being started, but the network utilization is only 100-140 MB/s. We tried a lot by playing with options like
- decompressor=“pigz -d”
but never come over 400 MB/sec with no load on the servers. After we copy the settings and add a new node, we come back to 100-140 MB/sec.
Can someone give me a tip what I can still try to use more network bandwidth?