Hi,
I’ve been evaluating Everest and testing it’s backup and restore functionality for MongoDB see how it behaves during disaster recovery. The backup and restore works without any issues as long as the database is up and running.
But when db is accidentally deleted, there is no way to restore the database from the backup in s3, (It’s fairly easy to accidentally delete the db since there are no non root users or deletion protection that can be enabled on db)
In order to test restoring from s3 when server is accidentally deleted, I deleted a db with backups and created a new db with the same name as deleted db and realised that backups from deleted db are visible in new db, but when I tried to restore the backup it’s stuck in a perpetual state of restoring with operator logs below
2024-02-25T08:52:25.358Z INFO Waiting for restore metadata {"controller": "psmdbrestore-controller", "object": {"name":"restore-i8h","namespace":"percona"}, "namespace": "percona", "name": "restore-i8h", "reconcileID": "181fbeb8-6730-4210-acba-64a87926fb74", "pbmName": "2024-02-25T07:44:50.291387568Z", "restore": "restore-i8h", "backup": "backup-t5p"}
2024-02-25T08:52:28.239Z INFO Waiting for restore to complete {"controller": "psmdbbackup-controller", "object": {"name":"cron-mongodb-4jy-20240225080000-f7nbc","namespace":"percona"}, "namespace": "percona", "name": "cron-mongodb-4jy-20240225080000-f7nbc", "reconcileID": "16de2ae5-3fb7-4507-96fe-343b1382a0cd", "restore": "restore-i8h", "status": "requested"}
2024-02-25T08:52:30.383Z INFO Waiting for restore metadata {"controller": "psmdbrestore-controller", "object": {"name":"restore-i8h","namespace":"percona"}, "namespace": "percona", "name": "restore-i8h", "reconcileID": "8e41a7e5-bbad-46a1-84b1-e080e6f14b7a", "pbmName": "2024-02-25T07:44:50.291387568Z", "restore": "restore-i8h", "backup": "backup-t5p"}
2024-02-25T08:52:33.253Z INFO Waiting for restore to complete {"controller": "psmdbbackup-controller", "object": {"name":"cron-mongodb-4jy-20240225080000-f7nbc","namespace":"percona"}, "namespace": "percona", "name": "cron-mongodb-4jy-20240225080000-f7nbc", "reconcileID": "bb0e9e97-9fbf-4355-b022-59144f62f4f8", "restore": "restore-i8h", "status": "requested"}
Everest provides a way to create a new db from a specific backup when the source db is up and running so I think a way to restore the db from s3 path is feasible and important for disaster recovery.
I hope adding this functionality is considered in future versions.
Everest version: 0.8.0
MongoDB version: 6.0.9-7
Best regards
Harsha