Sharded collections not responding to queries after mongodb restore

MongoDB Percona Backup for MongoDB
1

Hello All,

Looking for some help understanding a problem that is happening post psmdb-restore.

Setup Details => Sharded pmsdb running on K8s cluster with percona mongodb operator v1.15 version and PBM 2.3.0 for daily logical backup.

When we are trying to restore the backup to new setup… restore completes successfully. Restore Custom resource also gets in READY state but all the sharded collections from databases becomes non-responsive (e.g. db..find() hangs). If we manually patch timestamp to the “wanted” timestamp from log… in db.config.collections document for perticular collection, the collection becomes accessible.

Can someone please guide what might be leading to this behavior post restore. Adding below error from the logs…

{“t”:{“$date”:“2024-03-30T18:53:10.434+00:00”},“s”:“I”, “c”:“QUERY”, “id”:4625501, “ctx”:“conn266”,“msg”:“Unable to establish remote cursors”,“attr”:{“error”:{“code”:13388,“codeName”:“StaleConfig”,“errmsg”:“timestamp mismatch detected for test.testconfiguration”,“ns”:“test.testconfiguration”,“vReceived”:{“t”:{“$timestamp”:{“t”:1711794245,“i”:87}},“e”:{“$oid”:“6607e84590bd0e6c36065c9c”},“v”:{“$timestamp”:{“t”:1,“i”:1}}},“vWanted”:{“t”:{“$timestamp”:{“t”:1711819658,“i”:109}},“e”:{“$oid”:“66084b8a97b9c28ad9c95007”},“v”:{“$timestamp”:{“t”:1,“i”:1}}},“shardId”:“testrepset”},“nRemotes”:0}}

2

Hi @Vivek_Shinde !
Try to flush router config using

db.adminCommand(
   {
     flushRouterConfig: "<db.collection>"
   }
)