After a restore from a none Xtradb cluster backup to an Xtradb cluster using the percona operator I get a Too Many Connections error.
restore from backup wait as cluster is building the pods (3 replicas)
operator 1.14
“errorVerbose”: "Error 1040: Too many connections\nupdate root user password expiration policy
Documentation is confusing when talking about what users is needed for a successful cluster to get restored.
Hi @Freddy, you can use information from this doc How to use backups and asynchronous replication to move an external database to Kubernetes - Percona Operator for MySQL based on Percona XtraDB Cluster to understand how to make a backup from the source. It should work.
THank you for the reply @Slava_Sarzhan but I am still seeing errors for some clusters, not all of the clusters we have. This only happens on a new/restored cluster so must be something with the user access but I have checked and we have all of the 4 users that should be created. I just received this error about a root user access issue which is strange.
if I shell to any of the 3 pods I can use our root user to access mysql. Checking the secret it the correct password.
2024-03-18T14:54:51.436Z ERROR failed to create db instance {"controller": "pxc-controller", "namespace": "percona", "name": "db-cluster", "reconcileID": "edb20b3c-30af-45d7-85e4-4349202e248c", "error": "Error 1045 (28000) │
│ : Access denied for user 'root'@'10-42-13-170.percona-xtradb-cluster-operator.percona.svc.cluster' (using password: YES)"} │
│ github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).mysqlVersion │
│ /go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/version.go:426 │
│ github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).reconcileUsers │
│ /go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/users.go:94 │
│ github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).Reconcile │
│ /go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/controller.go:281 │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile │
│ /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:119 │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler │
│ /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:316 │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem │
│ /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:266 │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 │
│ /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:227
Is it some type of permission issue that I’m hitting… I keep getting these on a few of our cluster. @Slava_Sarzhan you have any idea what the issue could be?
is it because the root user was not updated/changed from the single vm percona mysql host? (before backup/restore)
{"log":"2024-03-20T23:41:26.031524Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:30.642669Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:34.583862Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.210728Z 10 [ERROR] [MY-010584] [Repl] Replica SQL: Error 'Operation ALTER USER failed for 'root'@'%'' on query. Default database: 'mysql'. Query: 'ALTER USER 'root'@'%' PASSWORD │
│ EXPIRE NEVER', Error_code: MY-001396\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.210808Z 10 [Warning] [MY-000000] [WSREP] Event 1 Query apply failed: 1, seqno 1879206\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.211738Z 0 [Note] [MY-000000] [Galera] Member 0(ym-qa1-tuidb-pxc-0) initiates vote on a945a4d6-dd7e-11ee-9d28-374a99cbf231:1879206,80e781a86a57e8c6: Operation ALTER USER failed │
│ for 'root'@'%', Error_code: 1396;\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.211990Z 0 [Note] [MY-000000] [Galera] Recomputed vote based on error codes: 1396. New vote edd4a152b546e2de will be used for further steps. Old Vote: 80e781a86a57e8c6\n","file": │
│ "/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.212020Z 0 [Note] [MY-000000] [Galera] Votes over a945a4d6-dd7e-11ee-9d28-374a99cbf231:1879206:\n edd4a152b546e2de: 1/3\nWaiting for more votes.\n","file":"/var/lib/mysql/mys │
│ qld-error.log"} │
│ {"log":"2024-03-20T23:41:39.212090Z 2 [Note] [MY-000000] [Galera] Got vote request for seqno a945a4d6-dd7e-11ee-9d28-374a99cbf231:1879206\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.214069Z 0 [Note] [MY-000000] [Galera] Member 1(ym-qa1-tuidb-pxc-2) initiates vote on a945a4d6-dd7e-11ee-9d28-374a99cbf231:1879206,80e781a86a57e8c6: Operation ALTER USER failed │
│ for 'root'@'%', Error_code: 1396;\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.214237Z 0 [Note] [MY-000000] [Galera] Recomputed vote based on error codes: 1396. New vote edd4a152b546e2de will be used for further steps. Old Vote: 80e781a86a57e8c6\n","file": │
│ "/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.214267Z 0 [Note] [MY-000000] [Galera] Votes over a945a4d6-dd7e-11ee-9d28-374a99cbf231:1879206:\n edd4a152b546e2de: 2/3\nWinner: edd4a152b546e2de\n","file":"/var/lib/mysql/my │
│ sqld-error.log"} │
│ {"log":"2024-03-20T23:41:39.214336Z 2 [Note] [MY-000000] [Galera] a945a4d6-dd7e-11ee-9d28-374a99cbf231:1879206 already voted on. Continue.\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:41:56.012547Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:42:21.509136Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:42:28.097071Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
│ {"log":"2024-03-20T23:42:48.965219Z 0 [Warning] [MY-000000] [Server] Too many connections\n","file":"/var/lib/mysql/mysqld-error.log"} │
Unanswered | Unsolved | Solved
MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright © 2006 - 2021 Percona LLC. All rights reserved.