Percona XtraDB Cluster on Kubernetes mysql operator (Point-In-Time Recovery) not happening

MySQL & MariaDB Percona Operator for MySQL
, , ,
12

@Sergey_Pronin - setting up allowUnsafeConfigurations set to true is intentional. Size from 3 to 1 is happening automatically upon restore regardless of spec.pitr.type=transaction/date/latest.

Sorry did not mentioned this earlier, below are pxc-operator logs when i run restore job:

{"level":"error","ts":1648640228.7463877,"caller":"pxc/controller.go:1139","msg":"sync users","error":"exec syncusers: command terminated with exit code 1 /  / ERROR 2005 (HY000): Unknown MySQL server host 'pxc-db-pxc-0.pxc-db-pxc.pxc.svc.cluster.local' (2)\nERROR (line:1893) : Could not find a primary cluster node\n","errorVerbose":"exec syncusers: command terminated with exit code 1 /  / ERROR 2005 (HY000): Unknown MySQL server host 'pxc-db-pxc-0.pxc-db-pxc.pxc.svc.cluster.local' (2)\nERROR (line:1893) : Could not find a primary cluster node\n\ngithub.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).syncPXCUsersWithProxySQL\n\t/go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/users.go:491\ngithub.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).resyncPXCUsersWithProxySQL.func1\n\t/go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/controller.go:1137\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1581","stacktrace":"github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).resyncPXCUsersWithProxySQL.func1\n\t/go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/controller.go:1139"}
{"level":"info","ts":1648640229.045958,"caller":"pxcrestore/controller.go:222","msg":"point-in-time recovering","cluster":"pxc-db"}
{"level":"info","ts":1648640229.61903,"caller":"pxc/controller.go:465","msg":"reconcile replication error","err":"get primary pxc pod: not found"}
{"level":"info","ts":1648640229.6228173,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640229.6228752,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640234.639589,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640234.6396563,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640240.72782,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640240.727888,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640243.1580393,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640243.1581097,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640246.8310893,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640246.8311534,"caller":"pxc/backup.go:87","msg":"Creating or updating backup job","name":"789e0-daily-backup","schedule":"0 0 * * *"}
{"level":"info","ts":1648640252.089586,"caller":"pxcrestore/controller.go:243","msg":"You can view xtrabackup log:\n$ kubectl logs job/restore-job-restore46-pxc-db\nIf everything is fine, you can cleanup the job:\n$ kubectl delete pxc-restore/restore46\n"}

FYI- I followed below procedure for restore as well there i have some issues too.
(please see my other question here (How to initialize a new cluster from a S3 backup? - #9 by ajay) where my restore from a latest backup is bringing up 3 nodes of PXC and im trying to target the specific date’s backup to restore but it always taking the latest and my latest backup dont have any databases that i manually imported that is why im targeting specific date’s backup. thoughts…)

1 Like