MySql cluster failed after catastrophic power failure and power-gen failed to feed datacenter

MySQL & MariaDB Percona XtraDB Cluster 5.x
3
  1. Hello Vadimtk,

    Thank you for reaching my post and replied this.

    However, I found the resolution to rebuild and repair cluster already by:

    Searching for the highest SQNO on all the nodes.

  2. Power-off all nodes except the highest SQNO one.
  3. Execute command: systemctl mysql@bootstrap.service on the said highest SQNO node.
  4. Wait...and cross your fingers, until the MySQL service on the bootstrap node is up.
  5. Power-on the rest nodes "one at a time" and keep eyes on the cluster sync status. By execute command: clustercheck and cat /var/lib/mysql/grastate.dat to check the GALERA saved state information, etc.

*** My steps above may wrong or not practical same as when the Linux experts investigate the same issue, but this is what I found and it works for my case. :slight_smile: ***

Thank you

Teerayuth