XtraDB operators on older version dont have the HAProxy and Cluster working properly

MySQL & MariaDB
1

Hi Team,

We are trying to install different version of Percona Operators in different namespace and found the following issue:

  1. Only the latest version of mysql Percona operator (1.10) have all the pods working as expected
  2. Older version 1.8 or 1.9 is having errors when running them

Please find the below screenshot from the GCP environment:

  1. On installation and execution of Version 8 Operator:
    $ kubectl config set-context $(kubectl config current-context) --namespace=siva-test-sql1-8
    Context “poc-admin@test-dbaas-poc” modified.
    [sivsiv@dbaasjump002 ~]$ kubectl get pods,svc
    NAME READY STATUS RESTARTS AGE
    pod/68e50-daily-backup-27355680-6jvnc 0/1 Completed 0 9h
    pod/68e50-sat-night-backup-27349920-rpmmb 0/1 Completed 0 4d9h
    pod/cluster1-haproxy-0 1/2 CrashLoopBackOff 4206 15d
    pod/cluster1-pxc-0 2/3 Running 1494 15d
    pod/percona-xtradb-cluster-operator-d99c748-pxpjr 1/1 Running 0 15d

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cluster1-haproxy ClusterIP 10.255.25.123 3306/TCP,3309/TCP,33062/TCP 15d
service/cluster1-haproxy-replicas ClusterIP 10.255.0.111 3306/TCP 15d
service/cluster1-pxc ClusterIP None 3306/TCP,33062/TCP 15d
service/cluster1-pxc-unready ClusterIP None 3306/TCP,33062/TCP 15d
service/percona-xtradb-cluster-operator ClusterIP 10.255.32.57 443/TCP 15d

  1. On installing the Version 9 of the Percona operator:
    $ kubectl config set-context $(kubectl config current-context) --namespace=siva-test-sql1-9
    Context “poc-admin@test-dbaas-poc” modified.
    $ kubectl get pods,svc
    NAME READY STATUS RESTARTS AGE
    pod/cluster1-haproxy-0 1/2 CrashLoopBackOff 539 47h
    pod/cluster1-pxc-0 2/3 Running 184 47h
    pod/percona-xtradb-cluster-operator-5bdd76bc45-bbfgh 1/1 Running 0 47h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cluster1-haproxy ClusterIP 10.255.43.215 3306/TCP,3309/TCP,33062/TCP,33060/TCP 47h
service/cluster1-haproxy-replicas ClusterIP 10.255.28.30 3306/TCP 47h
service/cluster1-pxc ClusterIP None 3306/TCP,33062/TCP,33060/TCP 47h
service/cluster1-pxc-unready ClusterIP None 3306/TCP,33062/TCP,33060/TCP 47h
service/percona-xtradb-cluster-operator ClusterIP 10.255.15.241 443/TCP 47h

  1. Only the Version 10 of the MySQL Percona Operator is working without issues:
    $ kubectl config set-context $(kubectl config current-context) --namespace=siva-test-sql1-10
    Context “poc-admin@test-dbaas-poc” modified.
    [sivsiv@dbaasjump002 ~]$ kubectl get pods,svc
    NAME READY STATUS RESTARTS AGE
    pod/cluster1-haproxy-0 2/2 Running 0 47h
    pod/cluster1-haproxy-1 2/2 Running 0 47h
    pod/cluster1-haproxy-2 2/2 Running 0 47h
    pod/cluster1-pxc-0 3/3 Running 0 47h
    pod/cluster1-pxc-1 3/3 Running 0 47h
    pod/cluster1-pxc-2 3/3 Running 0 47h
    pod/percona-xtradb-cluster-operator-566848cf48-xsvlb 1/1 Running 0 47h
    pod/xb-cron-cluster1-fs-pvc-202214005-372f8-2gpdj 0/1 Completed 0 33h
    pod/xb-cron-cluster1-fs-pvc-202215000-372f8-kfmv5 0/1 Completed 0 9h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cluster1-haproxy ClusterIP 10.255.30.154 3306/TCP,3309/TCP,33062/TCP,33060/TCP 47h
service/cluster1-haproxy-replicas ClusterIP 10.255.12.10 3306/TCP 47h
service/cluster1-pxc ClusterIP None 3306/TCP,33062/TCP,33060/TCP 47h
service/cluster1-pxc-unready ClusterIP None 3306/TCP,33062/TCP,33060/TCP 47h
service/percona-xtradb-cluster-operator ClusterIP 10.255.21.91 443/TCP 47h

Can you please check and update us why we are unable to use the older version of the operator once the newer version is released?

Thanks,
Sivapriya.

2

Hi @sivapriyas,

It’s hard to tell without seeing the haproxy and pxc logs but my gut feeling says you’re hitting the same issue in Fresh instance with Percona XtraDB Cluster Operator v1.8.0 not starting completly under OKD - #3 by mygov. The fix is landed in version 1.10.

1 Like
3

Thanks egegunes for your response - We see the similar behaviour in 1.09 as well - Also we observe only the latest version is working and the older version dont - The reason why we are asking - If we have the Prod setup with V1.9 and if it fails when the new version comes - Do we need to always upgrade or move to the latest version of the operator?

Please confirm.

Thanks,
Sivapriya

1 Like
4

You can upgrade to latest versions on your pace, the operator doesn’t force you to upgrade or start to fail if a new version released. This case was bug in the older versions and fix is released in version 1.10.

1 Like