Amazingly enough I can’t find information about how to do this. I have an 8 node cluster in a test environment. I want to eliminate 4 nodes from it. I figured I would just shut down 4 of them and be on my way. but the other four just keep spitting out errors into the error file trying to reconnect with those other four.
How do you cleanly remove a node from the cluster? I can’t find any documentation or commands about how to do it?
What does your wsrep_cluster_address -attribute look like in my.cnf of remaining nodes ? Is it possible that in remaining nodes are pointing to eliminated nodes ? Also you can check that wsrep attributes are identical in all nodes with show status like ‘wsrep%’;
If you had 8 nodes and “eliminated 4 nodes”, means it was not a clean service shutdown of those, remaining nodes detected split brain situation and therefore stopped working waiting for missing nodes to reconnect. Please read about Quorum in Galera.
Solution - stop the nodes with correct service stop command, ideally one by one.