PBM backup never finishing

Getting below errors.

$ pbm logs
2022-10-07T19:52:03Z I [sh_34/db31212.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mongodump finished, waiting for the oplog
2022-10-07T19:52:04Z I [sh_34/db31212.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] dropping tmp collections
2022-10-07T19:52:04Z I [sh_34/db31212.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mark RS as error `waiting for dump done: backup stuck, last beat ts: 1665166375`: <nil>
2022-10-07T19:52:04Z E [sh_34/db31212.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] backup: waiting for dump done: backup stuck, last beat ts: 1665166375
2022-10-07T19:52:18Z I [sh_0/db40619.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mongodump finished, waiting for the oplog
2022-10-07T19:52:19Z I [sh_0/db40619.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] dropping tmp collections
2022-10-07T19:52:19Z I [sh_0/db40619.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mark RS as error `waiting for dump done: backup stuck, last beat ts: 1665166375`: <nil>
2022-10-07T19:52:19Z E [sh_0/db40619.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] backup: waiting for dump done: backup stuck, last beat ts: 1665166375
2022-10-07T19:53:31Z I [sh_11/db40630.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mongodump finished, waiting for the oplog
2022-10-07T19:53:32Z I [sh_11/db40630.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] dropping tmp collections
2022-10-07T19:53:32Z I [sh_11/db40630.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mark RS as error `waiting for dump done: backup stuck, last beat ts: 1665166375`: <nil>
2022-10-07T19:53:32Z E [sh_11/db40630.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] backup: waiting for dump done: backup stuck, last beat ts: 1665166375
2022-10-07T19:54:49Z I [sh_28/db31268.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mongodump finished, waiting for the oplog
2022-10-07T19:54:50Z I [sh_28/db31268.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] dropping tmp collections
2022-10-07T19:54:50Z I [sh_28/db31268.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mark RS as error `waiting for dump done: backup stuck, last beat ts: 1665166375`: <nil>
2022-10-07T19:54:50Z E [sh_28/db31268.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] backup: waiting for dump done: backup stuck, last beat ts: 1665166375
2022-10-07T20:37:17Z I [sh_21/db31258.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mongodump finished, waiting for the oplog
2022-10-07T20:37:18Z I [sh_21/db31258.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] dropping tmp collections
2022-10-07T20:37:18Z I [sh_21/db31258.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] mark RS as error `waiting for dump done: backup stuck, last beat ts: 1665166375`: <nil>
2022-10-07T20:37:18Z E [sh_21/db31258.or1.omniture.com:27018] [backup/2022-10-07T18:12:39Z] backup: waiting for dump done: backup stuck, last beat ts: 1665166375

Currently running:

Snapshot backup “2022-10-07T18:12:39Z”, started at 2022-10-07T18:12:40Z. Status: error. [op id: 63406c17ba6952b20bb77581]

1 Like