ProxySQL Error: connection is locked to hostgroup 11 but trying to reach hostgroup 10

Logs:
[root@lvndev012336 kafka-aiops]# kubectl logs pod/cluster1-proxysql-0 -c proxysql -n do-kafka-v20 |grep error
[root@lvndev012336 kafka-aiops]# kubectl logs pod/cluster1-proxysql-1 -c proxysql -n do-kafka-v20 |grep error
2021-03-04 20:50:07 MySQL_Monitor.cpp:1845:monitor_galera_thread(): [ERROR] Got error. mmsd 0x7f4f3e0259c0 , MYSQL 0x7f4f3ba26e00 , FD 27 : timeout check
[root@lvndev012336 kafka-aiops]# kubectl logs pod/cluster1-proxysql-2 -c proxysql -n do-kafka-v20 |grep error
2021-03-04 20:50:07 MySQL_Monitor.cpp:1845:monitor_galera_thread(): [ERROR] Got error. mmsd 0x7f3816231400 , MYSQL 0x7f3814a10000 , FD 28 : timeout check

1 Like

Hello @saloni24 ,

so what happened here?
Could you please share:

  1. version of the operator you use
  2. your cr.yaml
  3. steps to reproduce this issue
1 Like

I am too getting this error.
Here is my CR:

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBCluster
metadata:
  finalizers:
  - delete-pxc-pods-in-order
  - delete-proxysql-pvc
  - delete-pxc-pvc
  generation: 1
  name: minidatapool
  namespace: ec-core
spec:
  allowUnsafeConfigurations: true
  backup:
    image: percona-xtradb-cluster-operator:1.7.0-pxc8.0-backup
    pitr:
      enabled: false
    schedule:
    - keep: 5
      name: monthly-backup
      schedule: 0 0 1 * *
      storageName: fs-pvc
    serviceAccountName: minidatapool-sa
    storages:
      fs-pvc:
        s3: null
        type: filesystem
        volume:
          emptyDir: null
          hostPath: null
          persistentVolumeClaim:
            accessModes:
            - ReadWriteOnce
            resources:
              requests:
                storage: 10Gi
  haproxy:
    affinity:
      advanced: null
      antiAffinityTopologyKey: none
    enabled: false
    gracePeriod: 30
    image: percona-xtradb-cluster-operator:1.7.0-haproxy
    podDisruptionBudget:
      maxUnavailable: 1
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    serviceAccountName: minidatapool-sa
    size: 1
  logcollector:
    enabled: true
    image: percona-xtradb-cluster-operator:1.7.0-logcollector
  pmm:
    enabled: false
    image: pmm-client:2.12.0
  proxysql:
    affinity:
      advanced: null
      antiAffinityTopologyKey: none
    enabled: true
    gracePeriod: 30
    image: percona-xtradb-cluster-operator:1.7.0-proxysql
    podDisruptionBudget:
      maxUnavailable: 1
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    serviceAccountName: minidatapool-sa
    size: 1
    volumeSpec:
      emptyDir: null
      hostPath: null
      persistentVolumeClaim:
        resources:
          requests:
            storage: 10Gi
  pxc:
    affinity:
      advanced: null
      antiAffinityTopologyKey: none
    enabled: true
    gracePeriod: 600
    image: percona-xtradb-cluster:8.0.21-12.1
    podDisruptionBudget:
      maxUnavailable: 1
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    serviceAccountName: minidatapool-sa
    size: 1
    volumeSpec:
      emptyDir: null
      hostPath: null
      persistentVolumeClaim:
        resources:
          requests:
            storage: 10Gi
  secretsName: minidatapool-secret
  sslInternalSecretName: my-cluster-ssl-internal
  sslSecretName: my-cluster-ssl

I have created the users following this link:
https://www.percona.com/doc/kubernetes-operator-for-pxc/users.html#users-unprivileged-users

I run into multiple issues when trying to connect to a database using one of the users created using above step. Sometimes I get access denined error when conecting using proxysql and sometimes I get the error reported in this thread.
However if I connect to pxc directy it works.

Is there a way to connext to proxysql pod and get more details on why it is not working or if users are getting synced to proxysql or not. Right now I am not able to connect to proxysql or I am not able to figure out how.
I tried following this document but it just not able to login to the mysql instance there to run the desired queries.
https://www.percona.com/doc/percona-xtradb-cluster/LATEST/howtos/proxysql-v2.html

1 Like

This is the additional logs from proxysql.

2021-05-27 06:39:01 MySQL_Session.cpp:6600:unable_to_parse_set_statement(): [WARNING] Unable to parse unknown SET query from client 10.7.4.21:37516. Setting lock_hostgroup. Please report a bug for future enhancements:SET foreign_key_checks=1, sql_safe_updates=0
2021-05-27 06:43:18 MySQL_Session.cpp:3941:handler(): [WARNING] Error during query on (10,minidatapool-pxc-0.minidatapool-pxc.ec-core.svc.cluster.local,3306,1632): 1142, SELECT command denied to user 'managerdb_9a12a062_a'@'minidatapool-proxysql-0.minidatapool-proxysql-unready.ec-core.sv' for table 'user_variables_by_thread'
1 Like

I’m get this error too.

1 Like

I think it is related to this bug: [K8SPXC-534] No servers in hostgroup 10 during pxc statefulset update - Percona JIRA

It is going to be resolved in 1.11 version of the Operator.

1 Like