Hello everyone,
I am using mysql cluster in master-master structure with three nodes. I started the service with the command “systemctl start mysql@bootstrap.service” on the first node, then I started the other two nodes. There is no problem in cluster synchronization, but when I look at the status on the first node, I see a fail statement. Can anyone help with knowledge of the cause?
root@prd-mysql8-01:~# systemctl status mysql
● mysql.service - Percona XtraDB Cluster
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2022-09-13 12:32:22 +03; 50min ago
Process: 1584 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)
Process: 1677 ExecStartPre=/usr/bin/mysql-systemd check-grastate (code=exited, status=1/FAILURE)
Process: 1734 ExecStopPost=/usr/bin/mysql-systemd stop-post (code=exited, status=0/SUCCESS)
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd systemd[1]: Starting Percona XtraDB Cluster…
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd mysql-systemd[1677]: WARNING: Node has been rebooted, /data/mysql/grastate.dat: seqno = -1, mysql service has not been started automatically
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd systemd[1]: mysql.service: Control process exited, code=exited, status=1/FAILURE
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd mysql-systemd[1734]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd mysql-systemd[1734]: WARNING: mysql may be already dead
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd systemd[1]: mysql.service: Failed with result ‘exit-code’.
Sep 13 12:32:22 prd-mysql8-01.tokenpay.prd systemd[1]: Failed to start Percona XtraDB Cluster.
root@prd-mysql8-02:~# systemctl status mysql
● mysql.service - Percona XtraDB Cluster
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2022-09-13 12:45:20 +03; 45min ago
Process: 3332 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)
Process: 3388 ExecStartPre=/usr/bin/mysql-systemd check-grastate (code=exited, status=0/SUCCESS)
Process: 3417 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 3419 ExecStartPre=/bin/sh -c VAR=bash /usr/bin/mysql-systemd galera-recovery; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
Process: 4406 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 4409 ExecStartPost=/usr/bin/mysql-systemd start-post $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 3467 (mysqld)
Status: “Server is operational”
Tasks: 66 (limit: 9405)
Memory: 587.2M
CGroup: /system.slice/mysql.service
└─3467 /usr/sbin/mysqld --wsrep_start_position=a9f9a7bf-3343-11ed-9186-276657d88acf:2
Sep 13 12:44:42 prd-mysql8-02.tokenpay.prd systemd[1]: Starting Percona XtraDB Cluster…
Sep 13 12:45:15 prd-mysql8-02.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4237, but reception only permitted for main PID 3467
Sep 13 12:45:15 prd-mysql8-02.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4281, but reception only permitted for main PID 3467
Sep 13 12:45:17 prd-mysql8-02.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4281, but reception only permitted for main PID 3467
Sep 13 12:45:17 prd-mysql8-02.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4281, but reception only permitted for main PID 3467
Sep 13 12:45:19 prd-mysql8-02.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4281, but reception only permitted for main PID 3467
Sep 13 12:45:20 prd-mysql8-02.tokenpay.prd mysql-systemd[4409]: SUCCESS!
Sep 13 12:45:20 prd-mysql8-02.tokenpay.prd systemd[1]: Started Percona XtraDB Cluster.
root@prd-mysql8-03:~# systemctl status mysql
● mysql.service - Percona XtraDB Cluster
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2022-09-13 12:46:29 +03; 19min ago
Process: 3235 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)
Process: 3290 ExecStartPre=/usr/bin/mysql-systemd check-grastate (code=exited, status=0/SUCCESS)
Process: 3320 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 3323 ExecStartPre=/bin/sh -c VAR=bash /usr/bin/mysql-systemd galera-recovery; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
Process: 4329 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 4334 ExecStartPost=/usr/bin/mysql-systemd start-post $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 3425 (mysqld)
Status: “Server is operational”
Tasks: 186 (limit: 9406)
Memory: 629.2M
CGroup: /system.slice/mysql.service
└─3425 /usr/sbin/mysqld --wsrep_start_position=a9f9a7bf-3343-11ed-9186-276657d88acf:3
Sep 13 12:45:53 prd-mysql8-03.tokenpay.prd systemd[1]: Starting Percona XtraDB Cluster…
Sep 13 12:46:24 prd-mysql8-03.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4163, but reception only permitted for main PID 3425
Sep 13 12:46:24 prd-mysql8-03.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4207, but reception only permitted for main PID 3425
Sep 13 12:46:25 prd-mysql8-03.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4207, but reception only permitted for main PID 3425
Sep 13 12:46:26 prd-mysql8-03.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4207, but reception only permitted for main PID 3425
Sep 13 12:46:27 prd-mysql8-03.tokenpay.prd systemd[1]: mysql.service: Got notification message from PID 4207, but reception only permitted for main PID 3425
Sep 13 12:46:29 prd-mysql8-03.tokenpay.prd mysql-systemd[4334]: SUCCESS!
Sep 13 12:46:29 prd-mysql8-03.tokenpay.prd systemd[1]: Started Percona XtraDB Cluster.