Kubernetes - [Galera] Handshake failed due to unknown CA

MySQL & MariaDB Percona Server for MySQL 8.0
1

Hello,
I am running the Percona operator for MySQL and PerconaXtradb on an EKS cluster, installed via the associated Helm charts (versions 1.12.0 and 1.13.0).

Earlier today I changed the values for the Xtradb chart and updated it (I’m trying to enable backups). The pod pxc-db-pxc-2 ends up in CrashLoopBackOff. When I look at the logs for the other two pods they’re full of TLS errors:
`
{“log”:“2023-10-03T22:00:00.459768Z 0 [Warning] [MY-000000] [Galera] Handshake failed: tlsv1 alert unknown ca\n”,“file”:“/var/lib/mysql/mysqld-error.log”}

{“log”:“2023-10-03T22:00:01.487659Z 0 [Warning] [MY-000000] [Galera] Handshake failed: tlsv1 alert unknown ca\n”,“file”:“/var/lib/mysql/mysqld-error.log”}

{“log”:“2023-10-03T22:00:02.966643Z 0 [Warning] [MY-000000] [Galera] Handshake failed: tlsv1 alert unknown ca\n”,“file”:“/var/lib/mysql/mysqld-error.log”}
`

This install is in a dev environment. It worked when it was set up, but hasn’t been touched since. Are there values that need to be given to Helm to configure the certs properly?

1 Like
2

I’m having exactly the same problem. It was working at the beginning, today I did some maintenance on some nodes (draining them and uncordoning them again afterwards), now I’m in the same situation, just that its pod0 having the issue.

Also deleting and therefore restarting all the pods, except the unhealthy one (!) it takes some time but the cluster gets back in sync and everything works for now.

The configuration in my case is pretty straight forward:

Operator without any values, defaults from helm:

project: database
source:
  repoURL: 'https://percona.github.io/percona-helm-charts/'
  targetRevision: 1.13.3
  helm:
    releaseName: mysql-operator
    values: '{}'
  chart: pxc-operator
destination:
  server: 'https://kubernetes.default.svc'
  namespace: database
syncPolicy:
  automated:
    prune: true
    selfHeal: true
  syncOptions:
    - CreateNamespace=true

Actual instance, just SC name overridden, nothing else.

project: database
source:
  repoURL: 'https://percona.github.io/percona-helm-charts/'
  targetRevision: 1.13.2
  helm:
    releaseName: mysqldb
    values: |-
      "pxc":
        "volumeSpec":
          "persistentVolumeClaim":
            "storageClassName": "rook-ceph-block"
  chart: pxc-db
destination:
  server: 'https://kubernetes.default.svc'
  namespace: database
syncPolicy:
  automated:
    prune: true
    selfHeal: true
  syncOptions:
    - CreateNamespace=true

Format is a bit weird, its ArgoCD applications deploying a helm chart.

3

We have the same behaviour (pxc-2 in crashloop and the other two pods complaining with a [Warning] [MY-000000] [Galera] Handshake failed: tlsv1 alert unknown ca\n","file":"/var/lib/mysql/mysqld-error.log"} error.

Any of you ever found out what the issue was?

4

Sadly I never figured it out and switched away from the Percona chart because of that.