Operator PS MySQL - behaviour on 1 instance that gets unavailable

MySQL & MariaDB Percona Operator for MySQL
1

Hi all,

just trying to understand the expected operator behaviour.

Description:

a cluster with 3 instances using Group Replication

  1. one pod/instance gets unavailable (network split 1 vs 2).
  2. It gets the state UNREACHABLE
  3. It gets removed from the cluster topology
  4. the pod is available again

In this situation seems that a Full Cluster Crash is “issued” causing downtime, can someone confirm that this is the expected behaviour? or the system should recover without downtime?

I would expect that a single instance failure/recovery would not require downtime (maybe i m doing something wrong).

Version:

1.2.0

Thanks!

(edit)
What i checked with some AI help:

Seems that the minority instance in network split (1 vs 2) when is not able to reach any member writes the /var/lib/mysql/full-cluster-crash.

This happens even if 2 of 3 members are healthy and running fine.

The operator trusts blindly on the minority instance that does not see the other 2 instances and runs the rebootClusterFromCompleteOutage.

I guess in this case the operator could/should confirm the state of the other instances to check if the cluster is or not healthy/running.

2

@Carlos_Abrantes

If one of the GR members is unreachable or down, the group should still remain intact. If the primary itself goes down, the Group Replication process triggers an automatic primary election, selecting one of the available members to become the new primary.

What was the Group Replication status among all active members?
select * from performance_schema.replication_group_members

Having group members’ pod-level logs and the mysqld error logs could provide better insight. Can you share those if feasible?