# Make mongodLockTimeout (30-min shutdown-checkpoint wait in physical/external restore) configurable **URL:** \ **Category:** Percona Backup for MongoDB\ **Tags:** percona, mongodb, pbm\ **Created:** [August 27, 2026, 9:52am UTC](https://forums.percona.com/t/make-mongodlocktimeout-30-min-shutdown-checkpoint-wait-in-physical-external-restore-configurable/41215 "2026-08-27T09:52:41Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Daniel\_Osuntoyinbo](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/daniel_osuntoyinbo/32/22879_2.png) [@Daniel\_Osuntoyinbo](https://forums.percona.com/u/Daniel_Osuntoyinbo)\ **Post date:** [August 27, 2026, 9:52am UTC](https://forums.percona.com/t/make-mongodlocktimeout-30-min-shutdown-checkpoint-wait-in-physical-external-restore-configurable/41215/1 "2026-08-27T09:52:41Z")
**Environment:** PSMDB (mongod 7.0.30-16), 3-node replica set, WiredTiger with `directoryForIndexes:true` Nodes: `r5.16xlarge`, data disk gp3 **16000 GiB / 40000 IOPS / 1000 MB/s** RAM: **248 GB WT cache** Dataset: 11 TB on-disk, one large index (`index/149-*.wt`, 1.7 TB) PBM version: **2.12.0** **Description** I kept hitting a wall while trying to restore a multi TB replicaset ( **11TB** ) with PBM, on both `pbm restore` (physical) and `pbm restore-finish` (external/EBS). Both go through `recoverStandaloneFromOplog()`, which replays the oplog in a temporary standalone mongod and then shuts it down. That shutdown forces a WiredTiger checkpoint, and PBM waits for it in `waitMgoShutdown()` using a hardcoded `mongodLockTimeout = 30 * time.Minute`. There’s no config, env var, or CLI flag to change it. Due to the huge data size / workload, the checkpoint needs more than 30 minutes to complete across a large number of WT index files, so the wait times out. PBM marks the restore `error`, and it tears down the data dir while the checkpoint thread is still writing causing `WT_PANIC` and leaves `/data` empty. I confirmed the checkpoint isn’t stuck as I patched the pbm-agent locally and raised the constant to **6h** locally, and the exact same ebs `restore-finish` phase completed successfully in about 1h45m. It would be good if this `mongodLockTimeout` parameter can be made configurable or the default parameter raised more generously so datasets at this scale can be restored successfully. see the code path here: [percona-backup-mongodb/pbm/restore/physical.go at v2.12.0 · percona/percona-backup-mongodb · GitHub](https://github.com/percona/percona-backup-mongodb/blob/v2.12.0/pbm/restore/physical.go#L59) Log snippets below: **timeout error** ```auto root@host-mongod-rs01-01:~# pbm describe-restore 2026-07-28T16:35:48.371868986Z -c /etc/pbm-storage.conf --mongodb-uri="mongodb://localhost:27018" name: "2026-07-28T16:35:48.371868986Z" opid: "" backup: "" type: physical status: error error: 'recover oplog as standalone: shutdown mongo: wait mongod shutdown: timeout during waiting for lock file /data/mongod.lock' start: "1970-01-01T00:00:00Z" last_transition_time: "2026-07-28T20:43:10Z" replsets: - name: rs01 status: error last_transition_time: "2026-07-28T20:43:10Z" nodes: - name: host-mongod-rs01-01.com:27018 status: error error: 'recover oplog as standalone: shutdown mongo: wait mongod shutdown: timeout during waiting for lock file /data/mongod.lock' last_transition_time: "2026-07-28T20:43:09Z" - name: host-mongod-rs01-02.com:27018 status: error error: 'recover oplog as standalone: shutdown mongo: wait mongod shutdown: timeout during waiting for lock file /data/mongod.lock' last_transition_time: "2026-07-28T20:42:23Z" - name: host-mongod-rs01-03.com:27018 status: error error: 'recover oplog as standalone: shutdown mongo: wait mongod shutdown: timeout during waiting for lock file /data/mongod.lock' last_transition_time: "2026-07-28T20:43:05Z" error: 'recover oplog as standalone: shutdown mongo: wait mongod shutdown: timeout during waiting for lock file /data/mongod.lock ``` **Physical restore failure, `pbm.restore.log`, ~11 TB, 2026-07-28 (exactly 30 min)** ```auto 20:11:50 temp standalone mongod started on port 27411 (recoverFromOplogAsStandalone=true, takeUnstableCheckpointOnShutdown=true, dbPath:/data) 20:11:52 ... 20:13:08 Completed oplog application for recovery ... numOpsApplied: 388147 (oplog replay SUCCEEDED, startup complete) 20:13:08 client application.name: pbm-physical-restore connects, issues {shutdown:1, timeoutMillis:15000} 20:13:08 "Shutting down checkpoint thread" → takeUnstableCheckpointOnShutdown forces full WT checkpoint 20:28..20:32 Checkpoint has been running for 1181 seconds and wrote: 505000 pages (9041 MB) (then NO further checkpoint progress logged for ~11 min) ~20:43:10 recover oplog as standalone: shutdown mongo: wait mongod shutdown: timeout during waiting for lock file /data/mongod.lock → restore marked error; PBM begins teardown 20:43:15 __directory_list_worker:46:/data/journal: opendir: No such file or directory → WT_PANIC → fassert → abort (PBM deleted dbpath while checkpoint thread still writing → /data empty afterward) ``` **after patching pbm-agent, restore completes successfully** ```auto pbm-agent: locally rebuilt from v2.12.0 with mongodLockTimeout changed 30m → 6h pbm restore --external → restore-finish index/149's shutdown checkpoint runs PAST the old 30-min kill point and COMPLETES successfully pbm describe-restore: status: done (all 3 nodes done, ~1h45m) ``` ---
**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\ **Post date:** [August 27, 2026, 12:11pm UTC](https://forums.percona.com/t/make-mongodlocktimeout-30-min-shutdown-checkpoint-wait-in-physical-external-restore-configurable/41215/2 "2026-08-27T12:11:52Z")
Hi, thank you for reaching out. I have created an improvement for this feature: [https://perconadev.atlassian.net/browse/PBM-1820](https://perconadev.atlassian.net/browse/PBM-1820) feel free to subscribe for updates.