In the last week we have had 4 outages to our production Percona XtraDB cluster, our cluster is a 5 node configuration, then all of a sudden the writes stop working and the connections spike on one node.
This causes a cluster wide outage. We have taken out the faulty node in the cluster and stopped networking to help troubleshoot, we can’t login to MySQL on the node, which is similar to what we experienced when it was in the cluster. The cluster is working, but we need to troubleshoot this node.
Could someone please help troubleshoot. we had the issue at 9:53pm, the fix was to stop networking on the faulty node and take it out of the cluster.