PSMDB 1.15.0 PITR Restore not working

MongoDB Percona Backup for MongoDB
,
1

Hello Team,

I am using PSMDB & Operator for MongoDB version 1.15.0 and trying to store and restore the PITR backups since incremental backup is not yet supported.
I have mentioned all the details related to issue on below thread.

Any inputs would help me to move ahead and stick to the decision of choosing Percona.

Regards,
Swwapnil Yadav

2

Same to me…

  1. config server, replicaset pod restarted without backup-agent sidecar (backup-agent logs is below. just shut down)
2024-01-02T05:38:48.000+0000 D [backup/2024-01-02T05:38:46Z] skip after nomination, probably started by another node
2024-01-02T05:38:52.000+0000 I [pitr] created chunk 2024-01-02T05:34:28 - 2024-01-02T05:38:42
2024-01-02T05:38:52.000+0000 I [pitr] pausing/stopping with last_ts 2024-01-02 05:38:42 +0000 UTC
2024-01-02T05:39:14.000+0000 D [pitr] start_catchup [oplog only]
2024-01-02T05:39:14.000+0000 D [pitr] lastTS set to {1704173922 2} 2024-01-02T05:38:42
2024-01-02T05:39:14.000+0000 I [pitr] streaming started from 2024-01-02 05:38:42 +0000 UTC / 1704173922
2024-01-02T05:39:14.000+0000 D [pitr] start_ok
2024-01-02T05:43:59.000+0000 I [pitr] got done signal, stopping
2024-01-02T05:44:01.000+0000 I [pitr] created chunk 2024-01-02T05:38:42 - 2024-01-02T05:43:52
2024-01-02T05:44:01.000+0000 I [pitr] pausing/stopping with last_ts 2024-01-02 05:43:52 +0000 UTC
2024/01/02 05:47:23 [entrypoint] got terminated, shutting down
2024/01/02 05:47:23 [entrypoint] kill `pbm-agent` (185): <nil>
  1. operator do : statefulset status check → run pbm config → fail (full log is below)
2024-01-02T05:48:35.258Z        INFO    Checking if replset is ready for physical restore       {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "2641eb55-4335-452c-9c5b-aace25c49d64", "replset": "cfg"}
2024-01-02T05:48:36.848Z        INFO    Replset is ready for physical restore   {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "2641eb55-4335-452c-9c5b-aace25c49d64", "replset": "cfg", "primary": "mongodb-test-psmdb-db-cfg-0"}
2024-01-02T05:48:36.848Z        INFO    Set PBM configuration   {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "2641eb55-4335-452c-9c5b-aace25c49d64", "command": ["/opt/percona/pbm", "config", "--file", "/etc/pbm/pbm_config.yaml"], "pod": "mongodb-test-psmdb-db-rs0-0"}
2024-01-02T05:48:42.284Z        INFO    Starting restore        {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "2641eb55-4335-452c-9c5b-aace25c49d64", "command": ["/opt/percona/pbm", "restore", "--base-snapshot", "2024-01-02T05:12:10Z", "--time", "2024-01-02T05:24:28", "--out", "json"], "pod": "mongodb-test-psmdb-db-rs0-0"}
2024-01-02T05:48:42.493Z        INFO    Restore started {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "2641eb55-4335-452c-9c5b-aace25c49d64", "pod": "mongodb-test-psmdb-db-rs0-0"}
2024-01-02T05:48:42.493Z        INFO    Restore state changed   {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "2641eb55-4335-452c-9c5b-aace25c49d64", "previous": "waiting", "current": "requested"}
2024-01-02T05:48:48.186Z        INFO    Restore state changed   {"controller": "psmdbrestore-controller", "object": {"name":"restore1","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "restore1", "reconcileID": "a344ba03-f440-458c-bdc8-d8ddc39bfac3", "previous": "requested", "current": "error"}

2024-01-02T05:50:14.085Z        INFO    balancer enabled        {"controller": "psmdb-controller", "object": {"name":"mongodb-test-psmdb-db","namespace":"mongodb-test"}, "namespace": "mongodb-test", "name": "mongodb-test-psmdb-db", "reconcileID": "969f2414-3230-4cee-92e1-664a5d01b9fe"}
3

It works to me now…
Do you have arbiter member in cluster?
And what os and image version? (server, backup agent)

4

Hi @kyeongjun.me

I am interested in your config because i cannot get restore of a pitr physical backup working. I tried to restore to a new kubernetes mongodb cluster without success.

Any hints are appreciated.

5

@dxzio Hi

Same to me… Restoring in new kubernetes cluster is not working for me.
My topic is here(link)
Uinsg cr backupName is working. but this case is that restoring in same kubernetes cluster.
Using backupSource(in aws s3 bucket) in new kubernetes cluster is not working.
You can see specific configure my upper topic

If I success, I will notice you :slight_smile:
Good luck for you.

6

@kyeongjun.me

Thank you for reply. I am wondering if the documentation Restore backup to a new Kubernetes-based environment - Percona Operator for MongoDB is working at all. And if yes with which percona mongodb operator version ? Maybe with the upcoming 1.16.0 ? Is there any bug ticket existing ?