# Forced reboot a node and doesn't start automatically **URL:** \ **Category:** Percona XtraDB Cluster 8.x\ **Created:** [February 16, 2026, 12:40pm UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217 "2026-02-16T12:40:51Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![chernandez\_a3sec](https://avatars.discourse-cdn.com/v4/letter/c/848f3c/32.png) [@chernandez\_a3sec](https://forums.percona.com/u/chernandez_a3sec)\ **Post date:** [February 16, 2026, 12:40pm UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/1 "2026-02-16T12:40:51Z")
Hi I’m testing a 3-node percona cluster and when a node is forced rebooted, mysql doesn’t start automatically. systemctl status mysql gives me: > mysql-systemd[618]: WARNING: Node has been rebooted, /var/lib/mysql/grastate.dat: seqno = -1, mysql service has not been started automatically Looking at the systemctl definition file, I see it calls to /usr/bin/mysql-systemd check-grastate Which does the following: > check\_grastate\_dat() { > local seqno=-1 > local uptime=$(awk ‘{print int($1/60)}’ /proc/uptime) > if [$uptime -lt 5]; then > if [-f $grastate\_loc]; then > seqno=$(grep ‘seqno:’ $grastate\_loc | cut -d: -f2 | tr -d ’ ') > if [$seqno -eq -1]; then > log\_warning\_msg “Node has been rebooted, $grastate\_loc: seqno = $seqno, mysql service has not been started automatically” > **exit 1** > fi > else > … So, if a node is forced rebooted, and uptime \< 5 minutes, systemctl start mysql throws an error. After 5 minutes, if I run the same command systemctl start mysql, starts with no problem (the warning is still logged). Why ? I assume the intention is the operator can check everything is ok and after 5 minutes, can start it manually. But if I want a completely automatic system (for example a forced reboot after a power failure) is it safe to remove the “exit 1” line so it can start automatically? Thanks ---
**Author:** ![jrivera](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/jrivera/32/13_2.png) [@jrivera](https://forums.percona.com/u/jrivera)\ **Post date:** [February 18, 2026, 6:55am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/2 "2026-02-18T06:55:00Z")
This is a safety gate to prevent unsafe automatic startup in an invalid Galera state. This was added in [PXC-2985](https://perconadev.atlassian.net/browse/PXC-2985). This is only blocked on unclean shutdowns. It is not advisable to remove `exit 1` in the script. ---
**Author:** ![chernandez\_a3sec](https://avatars.discourse-cdn.com/v4/letter/c/848f3c/32.png) [@chernandez\_a3sec](https://forums.percona.com/u/chernandez_a3sec)\ **Post date:** [February 18, 2026, 11:14am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/3 "2026-02-18T11:14:46Z")
> [@jrivera](#): > > PXC-2985 I’ve read it and I have the same question. When a node was forced rebooted, “systemctl start mysql” will fail the first 5 minutes of uptime. After that, it will start with no errors. Nothing happens in these first 5 minutes. Why is it safe to start it after 5 minutes? ---
**Author:** ![jrivera](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/jrivera/32/13_2.png) [@jrivera](https://forums.percona.com/u/jrivera)\ **Post date:** [February 19, 2026, 6:36am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/4 "2026-02-19T06:36:23Z")
It’s not that it’s already “safe” to start mysqld after 5 minutes has passed, but the manual mysqld startup by a DBA would avoid the following cases: - Letting a node auto-start and join a cluster, which may cause cluster inconsistencies - Avoid auto-starting nodes in cases where all members of the cluster were rebooted, as this may cause an endless restart loop Since the DBA manually restarted mysqld, the expectation is that they are aware that it is “safe” to start the node and let it join the cluster. ---
**Author:** ![chernandez\_a3sec](https://avatars.discourse-cdn.com/v4/letter/c/848f3c/32.png) [@chernandez\_a3sec](https://forums.percona.com/u/chernandez_a3sec)\ **Post date:** [February 19, 2026, 7:37am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/5 "2026-02-19T07:37:57Z")
hi. Thanks for the reply. I understand, but it doesn’t cover scenarios on a power failure and no DBA is doing anything. The idea is having a full 24x7 available, with no manual intervention, and in this case, when a power failure happens, the node will not start mysqld automatically and require a DBA starts it manually. Most (of almost all of them) high availability services cover this scenario, with no manual intervention. ---
**Author:** ![jrivera](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/jrivera/32/13_2.png) [@jrivera](https://forums.percona.com/u/jrivera)\ **Post date:** [February 24, 2026, 5:28am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/6 "2026-02-24T05:28:17Z")
Right, for cases like this, I believe the solution is to migrate to PXC Operator and let the Kubernetes controller and statefulset handle pod restarts. For on-premises or self-managed clusters, it is not advisable, because if all nodes restart abruptly almost simultaneously while DMLs are active on any given node, the DBA would need to start each member with `--wsrep_recover` to repair `grastate.dat`, determine which node has the latest commit, and start it as a bootstrapped node. Automatic service startup could start a member that is behind, resulting in either inconsistencies or data loss. Moreover, editing the `mysql-systemd` script would not scale and would require you to edit the file each time you upgrade. Your other option is to create a script that will start the service after an OS reboot if the other nodes are up and running. ---
**Author:** ![marno.krahmer](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@marno.krahmer](https://forums.percona.com/u/marno.krahmer)\ **Post date:** [May 13, 2026, 9:34am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/7 "2026-05-13T09:34:10Z")
Hey, I also stumbled across that. In my case, I manually checked and was sure, that it was safe to start MySQL again. (because it was a one-node-cluster for testing purposes only). I still could not do it, because even a manual “systemctl start mysql” is prevented within the first 5 minutes. Can you elaborate, how I would write a script to start the systemd-service without removing your 5-Minute-Limit? Or are you suggesting to start MySQL completely outside of systemd and implement our own process management? > Automatic service startup could start a member that is behind, resulting in either inconsistencies or data loss. Would mysql actually start with other nodes configured in `wsrep_cluster_address`, when none of these nodes is reachable and the node itself experienced an unclean shutdown? From my observation, a crashed node will have a grastate like: > # GALERA saved state > version\*:\* 2.1 > uuid\*:\* e256c10a-d2ea-11ed-96e2-c7a28cc52bd1 > seqno\*:\* -1 > safe\_to\_bootstrap\*:\* 0 And due to the “safe\_to\_bootstrap” being 0, the node knows, that it requires a state transfer from other nodes before starting. If all nodes crashed around the same time, then no node will be reachable for a state transfer and startup will fail. I can imagine how this can end up in a restart loop, but I don’t quite get, how this would end up in an unhealthy node ever entering “PRIMARY” state? ---
**Author:** ![Yunus](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/yunus/32/6430_2.png) [@Yunus](https://forums.percona.com/u/Yunus)\ **Post date:** [May 16, 2026, 1:19am UTC](https://forums.percona.com/t/forced-reboot-a-node-and-doesnt-start-automatically/40217/8 "2026-05-16T01:19:57Z")
> I also stumbled across that. In my case, I manually checked and was sure, that it was safe to start MySQL again. (because it was a one-node-cluster for testing purposes only). In one node cluster, you can force bootstrap that node and it should start. You should do this while starting the first node, ```auto systemctl start mysql@bootstrap.service ``` > Can you elaborate, how I would write a script to start the systemd-service without removing your 5-Minute-Limit? > Or are you suggesting to start MySQL completely outside of systemd and implement our own process management? You should check this document that shows to check all the cases when nodes shut down. > **[Percona XtraDB Cluster - Crash recovery](https://docs.percona.com/percona-xtradb-cluster/8.0/crash-recovery.html)** > > Unlike the standard MySQL replication, a PXC cluster acts like one logical > entity, which controls the status and consistency of each node as well as the > status of the whole cluster. This allows maintaining the data integrity more > efficiently than... The document should clear your all doubts regarding crashes and restarts.