# Percona Incremental backup - stuck at Waiting for the node to shutdown

**URL:** <https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307>\
**Category:** Percona Backup for MongoDB\
**Created:** [February 7, 2024, 6:57am UTC](https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307 "2024-02-07T06:57:23Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pjotz](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/pjotz/32/12032_2.png) [@Pjotz](https://forums.percona.com/u/Pjotz)\
**Post date:** [February 7, 2024, 6:57am UTC](https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307/1 "2024-02-07T06:57:24Z")

</div>

Hi  
Im trying to backup/restore a incremtental backup with percona. that backup runs like a charm, and if im running on a single node cluster, incremental restore works perfectly. but if i´m running in a sharded cluster, with 3x cfg servers, and 3 nodes, the restore starts but when getting to the step “waiting for the node to shutdown”, nothing happens, i can see the mongod service keeps running, but after around 30 sec, the pbm agent looses heartbeat… What is percona backup doing at that step? and what diffrent with single node and multinode in that step?

Logs:

> Starting the restore job:  
> mongodb@mongodb\_content\_cfg\_server2 /]$ pbm restore 2024-02-06T10:07:26Z -w  
> Starting restore 2024-02-07T06:47:59.304252431Z from ‘2024-02-06T10:07:26Z’…Started physical restore.  
> Waiting to finish…

> Logs from node:  
> {“t”:{“$date”:“2024-02-07T06:47:55.698+00:00”},“s”:“I”, “c”:“CONNPOOL”, “id”:22567, “ctx”:“ShardRegistry”,“msg”:“Ending idle connection because the pool meets constraints”,“attr”:{“hostAndPort”:“mongodb\_content\_rs1n1:27017”,“numOpenConns”:1}}  
> 2024-02-07T06:47:59.000+0000 I got command restore [name: 2024-02-07T06:47:59.304252431Z, snapshot: 2024-02-06T10:07:26Z] \<ts: 1707288479\>  
> 2024-02-07T06:47:59.000+0000 I got epoch {1707288344 4}  
> 2024-02-07T06:47:59.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] backup: 2024-02-06T10:07:26Z  
> 2024-02-07T06:47:59.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] recovery started  
> 2024-02-07T06:47:59.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] port: 27526  
> 2024-02-07T06:48:00.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] mongod binary: /usr/bin/mongod, version: v5.0.24-21  
> 2024-02-07T06:48:00.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] get src 2024-02-06T10:02:55Z  
> 2024-02-07T06:48:00.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] moving to state starting  
> 2024-02-07T06:48:00.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] waiting for `starting` status in rs map[.pbm.restore/2024-02-07T06:47:59.304252431Z/rs.mongo\_data\_rs2/node.mongodb\_content\_rs2n1:27017:{}]  
> {“t”:{“$date”:“2024-02-07T06:48:02.104+00:00”},“s”:“I”, “c”:“CONNPOOL”, “id”:22576, “ctx”:“ShardRegistry”,“msg”:“Connecting”,“attr”:{“hostAndPort”:“mongodb\_content\_cfg\_server3:27017”}}  
> {“t”:{“$date”:“2024-02-07T06:48:02.282+00:00”},“s”:“I”, “c”:“CONNPOOL”, “id”:22576, “ctx”:“ShardRegistry”,“msg”:“Connecting”,“attr”:{“hostAndPort”:“mongodb\_content\_cfg\_server1:27017”}}  
> 2024-02-07T06:48:05.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] waiting for cluster  
> 2024-02-07T06:48:15.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] converged to state starting  
> 2024-02-07T06:48:15.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] starting  
> 2024-02-07T06:48:15.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] moving to state running  
> 2024-02-07T06:48:15.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] waiting for `running` status in rs map[.pbm.restore/2024-02-07T06:47:59.304252431Z/rs.mongo\_data\_rs2/node.mongodb\_content\_rs2n1:27017:{}]  
> 2024-02-07T06:48:20.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] waiting for cluster  
> {“t”:{“$date”:“2024-02-07T06:48:29.643+00:00”},“s”:“I”, “c”:“STORAGE”, “id”:22430, “ctx”:“Checkpointer”,“msg”:“WiredTiger message”,“attr”:{“message”:“[1707288509:643870][7:0x7f3ed6126700], WT\_SESSION.checkpoint: [WT\_VERB\_CHECKPOINT\_PROGRESS] saving checkpoint snapshot min: 7058, snapshot max: 7058 snapshot count: 0, oldest timestamp: (1707288202, 1) , meta checkpoint timestamp: (1707288502, 1) base write gen: 84”}}  
> 2024-02-07T06:48:30.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] converged to state running  
> 2024-02-07T06:48:30.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] send to stopAgent chan  
> 2024-02-07T06:48:30.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] stop agents heartbeats  
> 2024-02-07T06:48:30.000+0000 I [restore/2024-02-07T06:47:59.304252431Z] stopping mongod and flushing old data  
> 2024-02-07T06:48:30.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] shutdown server  
> 2024-02-07T06:48:30.000+0000 D [restore/2024-02-07T06:47:59.304252431Z] waiting for the node to shutdown

> PBM status after 30 sec.  
> mongo\_data\_rs3:  
> mongo\_data\_rs3/mongodb\_content\_rs3n1:27017 [P]: pbm-agent FAILED status:  
> ERROR with ERROR: lost agent, last heartbeat: 1707288505  
> mongo\_data\_rs2:  
> mongo\_data\_rs2/mongodb\_content\_rs2n1:27017 [P]: pbm-agent FAILED status:  
> ERROR with ERROR: lost agent, last heartbeat: 1707288507  
> mongo\_data\_rs1:  
> mongo\_data\_rs1/mongodb\_content\_rs1n1:27017 [P]: pbm-agent FAILED status:  
> ERROR with ERROR: lost agent, last heartbeat: 1707288508  
> mongo\_conf:  
> mongo\_conf/mongodb\_content\_cfg\_server1:27017 [S]: pbm-agent FAILED status:  
> ERROR with ERROR: lost agent, last heartbeat: 1707288506  
> mongo\_conf/mongodb\_content\_cfg\_server2:27017 [S]: pbm-agent FAILED status:  
> ERROR with ERROR: lost agent, last heartbeat: 1707288506  
> mongo\_conf/mongodb\_content\_cfg\_server3:27017 [P]: pbm-agent FAILED status:  
> ERROR with ERROR: lost agent, last heartbeat: 1707288506

---

<div class="post-metadata">

**Author:** ![Pjotz](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/pjotz/32/12032_2.png) [@Pjotz](https://forums.percona.com/u/Pjotz)\
**Post date:** [February 8, 2024, 8:00am UTC](https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307/2 "2024-02-08T08:00:02Z")

</div>

I have tried to manually shutdown the mongod with /usr/bin/mongod --shutdown on all shards. then its starts to do the restore, but then stopping later on, and it seems like the restore is not fully successfull.

here are the logs:

> 2024-02-08T07:39:56.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] waiting for cluster  
> 2024-02-08T07:40:06.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] converged to state running  
> 2024-02-08T07:40:06.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] send to stopAgent chan  
> 2024-02-08T07:40:07.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] stop agents heartbeats  
> 2024-02-08T07:40:07.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] stopping mongod and flushing old data  
> 2024-02-08T07:40:07.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] shutdown server  
> 2024-02-08T07:40:07.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] waiting for the node to shutdown  
> {“t”:{“$date”:“2024-02-08T07:40:10.191+00:00”},“s”:“I”, “c”:“CONTROL”, “id”:23377, “ctx”:“SignalHandler”,“msg”:“Received signal”,“attr”:{“signal”:15,“error”:“Terminated”}}  
> {“t”:{“$date”:“2024-02-08T07:40:10.191+00:00”},“s”:“I”, “c”:“CONTROL”, “id”:23378, “ctx”:“SignalHandler”,“msg”:“Signal was sent by kill(2)”,“attr”:{“pid”:231,“uid”:1000}}  
> {“t”:{“$date”:“2024-02-08T07:40:10.191+00:00”},“s”:“I”, “c”:“CONTROL”, “id”:23381, “ctx”:“SignalHandler”,“msg”:“will terminate after current cmd ends”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.191+00:00”},“s”:“I”, “c”:“REPL”, “id”:4784900, “ctx”:“SignalHandler”,“msg”:“Stepping down the ReplicationCoordinator for shutdown”,“attr”:{“waitTimeMillis”:15000}}  
> {“t”:{“$date”:“2024-02-08T07:40:10.191+00:00”},“s”:“I”, “c”:“REPL”, “id”:4794602, “ctx”:“SignalHandler”,“msg”:“Attempting to enter quiesce mode”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.192+00:00”},“s”:“I”, “c”:“COMMAND”, “id”:4784901, “ctx”:“SignalHandler”,“msg”:“Shutting down the MirrorMaestro”}  
> … (removed because of to many chars in post) “ctx”:“SignalHandler”,“msg”:“shutdown: removing fs lock…”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.382+00:00”},“s”:“I”, “c”:“-”, “id”:4784931, “ctx”:“SignalHandler”,“msg”:“Dropping the scope cache for shutdown”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.382+00:00”},“s”:“I”, “c”:“FTDC”, “id”:4784926, “ctx”:“SignalHandler”,“msg”:“Shutting down full-time data capture”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.382+00:00”},“s”:“I”, “c”:“FTDC”, “id”:20626, “ctx”:“SignalHandler”,“msg”:“Shutting down full-time diagnostic data capture”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.388+00:00”},“s”:“I”, “c”:“CONTROL”, “id”:20565, “ctx”:“SignalHandler”,“msg”:“Now exiting”}  
> {“t”:{“$date”:“2024-02-08T07:40:10.388+00:00”},“s”:“I”, “c”:“CONTROL”, “id”:23138, “ctx”:“SignalHandler”,“msg”:“Shutting down”,“attr”:{“exitCode”:0}}  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] revome old data  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-21-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-12-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-25–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-2–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-5-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-0–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-24-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/\_mdb\_catalog.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-26-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-15-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-16–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-37-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-12–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-18–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-21–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/sizeStorer.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-34–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/WiredTiger.wt  
> … (removed because of to many chars in post)  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-33-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/storage.bson  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-3-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-5–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-15–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-35-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-8–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/mongod.lock  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-17-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/journal  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-23-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-25-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-38-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-8-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-6–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-14-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-11-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-26–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/WiredTiger  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-39-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-18-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-9–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-41-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-42-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/WiredTiger.lock  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-13-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-7–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-11–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-17–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-43-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/diagnostic.data  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-1-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-13–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-1–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-14–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-24–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-6-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-29-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-27-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-29–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-31-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-19–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-34-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-4-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-2-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-35–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-7-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-0-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-22-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-22–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-10–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-28-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-40-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-3–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/index-19-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-4–3129342514983500851.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-36-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] remove /data/db/collection-9-8613182035066973332.wt  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copying backup data  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/journal/WiredTigerLog.0000000004.s2.0-104857600\> to \</data/db/journal/WiredTigerLog.0000000004\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/WiredTigerHS.wt.s2.0-24576\> to \</data/db/WiredTigerHS.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/\_mdb\_catalog.wt.s2.0-36864\> to \</data/db/\_mdb\_catalog.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-0–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-0–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-0-8613182035066973332.wt.s2.0-4096\> to \</data/db/collection-0-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-10–3129342514983500851.wt.s2.0-20480\> to \</data/db/collection-10–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-12–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-12–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-12-8613182035066973332.wt.s2.0-20480\> to \</data/db/collection-12-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-14–3129342514983500851.wt.s2.0-999424\> to \</data/db/collection-14–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-15–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-15–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-17–3129342514983500851.wt.s2.0-20480\> to \</data/db/collection-17–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-18-8613182035066973332.wt.s2.0-24576\> to \</data/db/collection-18-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-19–3129342514983500851.wt.s2.0-32768\> to \</data/db/collection-19–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-2–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-2–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-21–3129342514983500851.wt.s2.0-4096\> to \</data/db/collection-21–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-21-8613182035066973332.wt.s2.0-20480\> to \</data/db/collection-21-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-23–3129342514983500851.wt.s2.0-4096\> to \</data/db/collection-23–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-25–3129342514983500851.wt.s2.0-20480\> to \</data/db/collection-25–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-27–3129342514983500851.wt.s2.0-24576\> to \</data/db/collection-27–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-27-8613182035066973332.wt.s2.0-4096\> to \</data/db/collection-27-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-29–3129342514983500851.wt.s2.0-24576\> to \</data/db/collection-29–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-3-8613182035066973332.wt.s2.0-20480\> to \</data/db/collection-3-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-30-8613182035066973332.wt.s2.0-20480\> to \</data/db/collection-30-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-34–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-34–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-36-8613182035066973332.wt.s2.0-135168\> to \</data/db/collection-36-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-39-8613182035066973332.wt.s2.0-36864\> to \</data/db/collection-39-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-4–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-4–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-42-8613182035066973332.wt.s2.0-28672\> to \</data/db/collection-42-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-6–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-6–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-8–3129342514983500851.wt.s2.0-36864\> to \</data/db/collection-8–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/collection-9-8613182035066973332.wt.s2.0-4096\> to \</data/db/collection-9-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-1–3129342514983500851.wt.s2.0-36864\> to \</data/db/index-1–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-1-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-1-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-10-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-10-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-11–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-11–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-11-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-11-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-13–3129342514983500851.wt.s2.0-36864\> to \</data/db/index-13–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-13-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-13-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-14-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-14-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-15-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-15-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-16–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-16–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-16-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-16-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-17-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-17-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-18–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-18–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-19-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-19-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-2-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-2-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-20–3129342514983500851.wt.s2.0-12288\> to \</data/db/index-20–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-20-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-20-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-22–3129342514983500851.wt.s2.0-4096\> to \</data/db/index-22–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-22-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-22-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-23-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-23-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-24–3129342514983500851.wt.s2.0-4096\> to \</data/db/index-24–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-24-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-24-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-25-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-25-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-26–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-26–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-26-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-26-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-28–3129342514983500851.wt.s2.0-24576\> to \</data/db/index-28–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-28-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-28-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-29-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-29-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-3–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-3–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-30–3129342514983500851.wt.s2.0-24576\> to \</data/db/index-30–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-31-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-31-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-32-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-32-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-33-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-33-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-34-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-34-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-35–3129342514983500851.wt.s2.0-36864\> to \</data/db/index-35–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-35-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-35-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-37-8613182035066973332.wt.s2.0-94208\> to \</data/db/index-37-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-38-8613182035066973332.wt.s2.0-86016\> to \</data/db/index-38-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-4-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-4-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-40-8613182035066973332.wt.s2.0-36864\> to \</data/db/index-40-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-41-8613182035066973332.wt.s2.0-36864\> to \</data/db/index-41-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-43-8613182035066973332.wt.s2.0-28672\> to \</data/db/index-43-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-5–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-5–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-5-8613182035066973332.wt.s2.0-20480\> to \</data/db/index-5-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-6-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-6-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-7–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-7–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-7-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-7-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-8-8613182035066973332.wt.s2.0-4096\> to \</data/db/index-8-8613182035066973332.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/index-9–3129342514983500851.wt.s2.0-20480\> to \</data/db/index-9–3129342514983500851.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/sizeStorer.wt.s2.0-36864\> to \</data/db/sizeStorer.wt\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/WiredTiger.backup.s2.0-153882\> to \</data/db/WiredTiger.backup\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/WiredTiger.s2.0-50\> to \</data/db/WiredTiger\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/storage.bson.s2.0-114\> to \</data/db/storage.bson\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] copy \<2024-02-08T06:24:57Z/mongo\_data\_rs3/journal/WiredTigerLog.0000000005.s2\> to \</data/db/journal/WiredTigerLog.0000000005\>  
> 2024-02-08T07:40:11.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] preparing data  
> 2024-02-08T07:40:13.000+0000 D [restore/2024-02-08T07:39:40.840706376Z] oplogTruncateAfterPoint: {1707373499 2}  
> 2024-02-08T07:40:14.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] recovering oplog as standalone  
> 2024-02-08T07:40:16.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] clean-up and reset replicaset config  
> 2024-02-08T07:40:19.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] restore on node succeed  
> 2024-02-08T07:40:19.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] moving to state done  
> 2024-02-08T07:40:19.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] waiting for `done` status in rs map[.pbm.restore/2024-02-08T07:39:40.840706376Z/rs.mongo\_data\_rs3/node.mongodb\_content\_rs3n1:27017:{}]  
> 2024-02-08T07:40:24.000+0000 I [restore/2024-02-08T07:39:40.840706376Z] waiting for shards map[.pbm.restore/2024-02-08T07:39:40.840706376Z/rs.mongo\_conf/rs:{} .pbm.restore/2024-02-08T07:39:40.840706376Z/rs.mongo\_data\_rs1/rs:{} .pbm.restore/2024-02-08T07:39:40.840706376Z/rs.mongo\_data\_rs2/rs:{} .pbm.restore/2024-02-08T07:39:40.840706376Z/rs.mongo\_data\_rs3/rs:{}]

> then its stuck here.  
> any idea whats going wrong? i will paste my pbm\_config.yaml also

> storage:  
> type: filesystem  
> filesystem:  
> path: /opt/XXX/backup/  
> restore:  
> mongodLocation: /usr/bin/mongod  
> mongodLocationMap:  
> “mongodb\_content\_cfg\_server1:27017”: /usr/bin/mongod  
> “mongodb\_content\_cfg\_server2:27017”: /usr/bin/mongod  
> “mongodb\_content\_cfg\_server3:27017”: /usr/bin/mongod  
> “mongodb\_content\_rs1n1:27017”: /usr/bin/mongod  
> “mongodb\_content\_rs2n1:27017”: /usr/bin/mongod  
> “mongodb\_content\_rs3n1:27017”: /usr/bin/mongod

---

<div class="post-metadata">

**Author:** ![Pjotz](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/pjotz/32/12032_2.png) [@Pjotz](https://forums.percona.com/u/Pjotz)\
**Post date:** [February 8, 2024, 1:29pm UTC](https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307/3 "2024-02-08T13:29:16Z")

</div>

update:  
I dont have any auth on my mongodb, is that an issue? i can see i cannot remotely run the db.runcommand({shutdown: 1}) because no auth is enabled, but the question is. Are the pbm-agent running the shutdown command from localhost, or is it running the command remotely from the cfg server where the restore is executed?

can you please help out here? 🙂

---

<div class="post-metadata">

**Author:** ![Pjotz](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/pjotz/32/12032_2.png) [@Pjotz](https://forums.percona.com/u/Pjotz)\
**Post date:** [February 9, 2024, 9:54am UTC](https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307/4 "2024-02-09T09:54:46Z")

</div>

update:  
I have now enabled auth on my mongodb, and the restore is now running successfull. so it seems like it connects from the cfg server to the other hosts and running the shutdown, is this the correct assumption?

so question is now… is there anyway to get this running without authentication?

/Morten

---

<div class="post-metadata">

**Author:** ![Pjotz](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/pjotz/32/12032_2.png) [@Pjotz](https://forums.percona.com/u/Pjotz)\
**Post date:** [February 9, 2024, 10:54am UTC](https://forums.percona.com/t/percona-incremental-backup-stuck-at-waiting-for-the-node-to-shutdown/28307/5 "2024-02-09T10:54:23Z")

</div>

Update again…

Found out that my pbm-uri had my hostname and not localhost, after changing to localhost, everything now works…  
Hopefully this thread can be helpfull for others, trying to run mongodb and pbm in same docker container.