Mysql is failed but synced

HI, my problem (or not) is:

/etc/init.d/mysql status
● mysql.service - LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon
Loaded: loaded (/etc/init.d/mysql)
Active: failed (Result: timeout) since Qua 2016-10-19 12:36:56 WEST; 23min ago

Out 19 12:36:56 systemd[1]: mysql.service start operation timed out. Terminating.
Out 19 12:36:56 systemd[1]: Failed to start LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon.
Out 19 12:36:56 systemd[1]: Unit mysql.service entered failed state.
Out 19 12:36:58 mysql[30262]: Starting MySQL (Percona XtraDB Cluster) database server: mysqld . .State transf… . . .
Hint: Some lines were ellipsized, use -l to show in full.

But the node is synced with other 2.

2016-10-19 12:39:11 30915 [Note] WSREP: 2.0 (node5): State transfer from 0.0 () complete.
2016-10-19 12:39:11 30915 [Note] WSREP: Shifting JOINER → JOINED (TO: 17959590)
2016-10-19 12:39:11 30915 [Note] WSREP: Member 2.0 (node5) synced with group.
2016-10-19 12:39:11 30915 [Note] WSREP: Shifting JOINED → SYNCED (TO: 17959590)
2016-10-19 12:39:11 30915 [Note] WSREP: Synchronized with group, ready for connections

What’s happening here?

Thanks.

Can you share the complete log file for the said node.

I am having the same thing
the Primary node says its failed … but its running … all other nodes come back as actively running
I am assuming this is from syslogs first run ?

on one note can some code be put in place to remove the pid ?

  • The server quit without updating PID file (/var/lib/mysql/ubuntuServer.pid).
    Nov 19 16:30:20 ubuntuServer mysql[620]: …fail!

its a pain manually deleting the pid to restart …

on main node
/etc/init.d/mysql bootstrap-pxc

root@ubuntuServer:/var/lib/mysql# systemctl status mysql.service
● mysql.service - LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon
Loaded: loaded (/etc/init.d/mysql; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2016-11-19 16:30:20 AEST; 12h ago
Docs: man:systemd-sysv-generator(8)
Process: 620 ExecStart=/etc/init.d/mysql start (code=exited, status=1/FAILURE)

Nov 19 16:29:44 ubuntuServer systemd[1]: Starting LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon…
Nov 19 16:29:45 ubuntuServer mysql[620]: * Starting MySQL (Percona XtraDB Cluster) database server mysqld
Nov 19 16:29:45 ubuntuServer /etc/init.d/mysql[662]: MySQL PID not found, pid_file detected/guessed: /var/lib/mysql/ubuntuServer.pid
Nov 19 16:30:20 ubuntuServer mysql[620]: * The server quit without updating PID file (/var/lib/mysql/ubuntuServer.pid).
Nov 19 16:30:20 ubuntuServer mysql[620]: …fail!
Nov 19 16:30:20 ubuntuServer systemd[1]: mysql.service: Control process exited, code=exited status=1
Nov 19 16:30:20 ubuntuServer systemd[1]: Failed to start LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon.
Nov 19 16:30:20 ubuntuServer systemd[1]: mysql.service: Unit entered failed state.
Nov 19 16:30:20 ubuntuServer systemd[1]: mysql.service: Failed with result ‘exit-code’.

yet the other nodes come back as running using systemctl status? how would I see the bootstrapped details ?
the only process I can see running are

1280 mysql 20 0 1363M 423M 19864 S 0.0 42.7 0:01.13 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=
1031 root 20 0 4508 1464 1464 S 0.0 0.1 0:00.01 /bin/sh /usr/bin/mysqld_safe --wsrep-new-cluster

systemctl status mysqld_safe.service
● mysqld_safe.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)

root@ubuntuServer:~# systemctl status mysqld.service
● mysqld.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)

The other nodes

/etc/init.d/mysql start
[ ok ] Starting mysql (via systemctl): mysql.service.

root@ubuntuServerN1:/var/lib/mysql# /etc/init.d/mysql start
[ ok ] Starting mysql (via systemctl): mysql.service.
root@ubuntuServerN1:/var/lib/mysql# systemctl status mysql.service
● mysql.service - LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon
Loaded: loaded (/etc/init.d/mysql; bad; vendor preset: enabled)
Active: active (running) since Sun 2016-11-20 05:13:23 AEST; 1min 0s ago
Docs: man:systemd-sysv-generator(8)
Process: 1347 ExecStart=/etc/init.d/mysql start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/mysql.service
├─1386 /bin/sh /usr/bin/mysqld_safe
└─1650 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --wsrep-pr

Nov 20 05:12:14 ubuntuServerN1 systemd[1]: Starting LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon…
Nov 20 05:12:14 ubuntuServerN1 mysql[1347]: * Starting MySQL (Percona XtraDB Cluster) database server mysqld
Nov 20 05:12:14 ubuntuServerN1 /etc/init.d/mysql[1385]: MySQL PID not found, pid_file detected/guessed: /var/lib/mysql/ubuntuServerN1
Nov 20 05:12:22 ubuntuServerN1 mysql[1347]: * State transfer in progress, setting sleep higher mysqld
Nov 20 05:13:23 ubuntuServerN1 mysql[1347]: …done.
Nov 20 05:13:23 ubuntuServerN1 systemd[1]: Started LSB: Start and stop the mysql (Percona XtraDB Cluster) daemon.

everything appears to be working how ever

mysql> show status like ‘wsrep%’;