Moving Datacenter

[Giovanni_Vecchi]

Hi guys,

our current scenario is the following:
- 5 nodes PXC 
- DC1 (AWS): 3 nodes spanned on 3 different AZ
- DC2 (on-prem): 2 nodes

DC2 is going to be decomissioned in order to deploy the new one DC3 in a different location, with different hardware and different subnets.
DC3 will come up after DC2 complete decomissioning, so I need to reduce PXC Cluster removing DC2 nodes and then put 2 DC3 nodes into the cluster sequentially.
What’s the best practices to do that with less downtime possible? Do I need to completely shutdown the cluster in order ro reduce it?

Thanks

[Giovanni_Vecchi]

I think I could answer to my own question after some studying and test sessions: the cluster does not need to completely shutdown and nodes can be removed and added as soon as quorum is kept on DC1 and mysql is graceful shutdowned on decomissioned nodes.
The only document I found where the safely nodes removal is clearly documented is the MariaDB cookbook:

Dropping a node from MariaDB Galera Cluster
MariaDB Galera Cluster exists so that we can eliminate single points of failure in our infrastructure. Once set up, a single node can be taken out of the cluster for maintenance without impacting the rest of the cluster or causing downtime for our applications.
…
1. On db03, run the following command to check whether the node is up to date:
  mysql -e “SHOW STATUS LIKE ‘wsrep_local_state_comment’”
2. If the value is synced, then the node is up to date and we can safely shut down thenode with the following command:
  sudo   service mysql stop
3. On the other cluster nodes, run the following command to check whether the nodehas been destroyed:
  mysql -e “SHOW STATUS LIKE ‘wsrep_%’”
4. We’ll know the node is destroyed if the number of nodes in wsrep_cluster_size is 2 and the IP address of db03 is not listed in wsrep_incoming_addresses.