Can't Start My Cluster :( It's So Urgent

MySQL & MariaDB Percona XtraDB Cluster 5.x
1

Hi guys,
I have a 3 Node cluster using XTraDb Cluster, all machines got shut down because of power failure
Now I can’t start Cluster, No matter which node I try

service mysql@bootstrap start

I get same error



-- Subject: Unit mysql@bootstrap.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mysql@bootstrap.service has begun starting up.
Aug 02 12:08:43 mysql1 mysql-systemd[3565]: State transfer in progress, setting sleep higher
Aug 02 12:08:43 mysql1 mysqld_safe[3564]: 2017-08-02T12:08:43.550234Z mysqld_safe Logging to '/var/log/mysqld.log'.
Aug 02 12:08:43 mysql1 mysqld_safe[3564]: 2017-08-02T12:08:43.554885Z mysqld_safe Logging to '/var/log/mysqld.log'.
Aug 02 12:08:43 mysql1 mysqld_safe[3564]: 2017-08-02T12:08:43.601278Z mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Aug 02 12:08:43 mysql1 mysqld_safe[3564]: 2017-08-02T12:08:43.617033Z mysqld_safe Skipping wsrep-recover for 73bddfbd-71e0-11e7-b0e0-4e3d5b56006f:31 pair
Aug 02 12:08:43 mysql1 mysqld_safe[3564]: 2017-08-02T12:08:43.619704Z mysqld_safe Assigning 73bddfbd-71e0-11e7-b0e0-4e3d5b56006f:31 to wsrep_start_position
Aug 02 12:08:48 mysql1 mysqld_safe[3564]: 2017-08-02T12:08:48.866922Z mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Aug 02 12:08:53 mysql1 mysql-systemd[3565]: /usr/bin/mysql-systemd: line 140: kill: (3564) - No such process
Aug 02 12:08:53 mysql1 mysql-systemd[3565]: ERROR! mysqld_safe with PID 3564 has already exited: FAILURE
Aug 02 12:08:53 mysql1 systemd[1]: mysql@bootstrap.service: control process exited, code=exited status=1
Aug 02 12:08:53 mysql1 mysql-systemd[4193]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Aug 02 12:08:53 mysql1 mysql-systemd[4193]: ERROR! mysql already dead
Aug 02 12:08:53 mysql1 systemd[1]: mysql@bootstrap.service: control process exited, code=exited status=2
Aug 02 12:08:53 mysql1 mysql-systemd[4223]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Aug 02 12:08:53 mysql1 mysql-systemd[4223]: WARNING: mysql may be already dead
Aug 02 12:08:53 mysql1 systemd[1]: Failed to start Percona XtraDB Cluster with config /etc/sysconfig/mysql.bootstrap.
-- Subject: Unit mysql@bootstrap.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mysql@bootstrap.service has failed.
-- 
-- The result is failed.
Aug 02 12:08:53 mysql1 systemd[1]: Unit mysql@bootstrap.service entered failed state.
Aug 02 12:08:53 mysql1 systemd[1]: mysql@bootstrap.service failed.
Aug 02 12:08:53 mysql1 polkitd[515]: Unregistered Authentication Agent for unix-process:3504:37712 (system bus name :1.15, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)

Could you help me Please, It’s Urgent :frowning:

2

Hi, have you tried the grastate.dat on each node - whether any of them has

safe_to_bootstrap: 1

?
If not, you have to first check each node position, with

mysqld_safe --wsrep-recover

and see which one is most advanced. Then, edit the grastate.dat to enable bootstrap.

If still failure, attach error logs.

3

Thank you Thank you Thank you Thank you :):):):slight_smile: <3
All nodes had 0 in safe_to_bootstrap, Since i have one write node and two read ( using proxysql ) , I know my first node is always the Master Node ( Checked in Proxysql to see who was write node before restart ),
Anyway I checked all and switched safe_to_bootstrap to 1 in my master node, Started Fine

Thanks Again

4

drupalshift; on behalf of my colleague at Percona - you’re welcome! May we suggest having a look at our support offerings to help you with future encounters or issues? Please see [url]https://www.percona.com/store[/url]