MySQL StatefulSet can't rollout to ready state after cluster spec update

MySQL & MariaDB Percona Operator for MySQL
,
1

github issue

Report

Cluster can’t transit to ready state after any change at .spec which lead to mysql StatefulSet rollout that can’t complete.

More about the problem

I deployed mysql async replication cluster with two replicas and without backups via default helm chart values. After cluster had reached ready state I turned on backups and rollout of mysql StatefulSet began from mysql-1 pod to mysql-0, but the last one stuck permanently with a problem at mysql container - “Startup probe failed” (Back-off restarting failed container)

Steps to reproduce

  1. deploy async replication cluster with 2 replicas and without backups section - backup.enabled: false
  2. wait for cluster ready state
  3. turn on backups - backup.enabled: true and fill this section with some meaningful values or you can just change mysql resources section - any action fits which lead to mysql StateFulSet rollout

Versions

  1. Kubernetes: 1.23.17
  2. Operator: percona/percona-server-mysql-operator:0.6.0
  3. Database: percona/percona-server:8.0.33-25

Anything else?

If I manually delete mysql statefulset it will fix the problem, at case of group replication there is no such a problem at all.

2

Hello @SlavaUtesinov .

Few comments:

  1. This operator is in alpha stage, we recommend using the one based on PXC. I assume you have some reason to run async replication and would be curious to learn more. Please share the use case.
  2. k8s version seems to be pretty old (1.23), we did not test on it
  3. Do you see any errors in the operator log?