Steps to Backup only OPlog Slice

Hello Team,

Query 1 :
The backing up of OPLog slice is possible with Base backup - Physical & Logical backup ?

Query 2 :
I am using PBM - 2.0.2. I wish to backup only OPlog slice & restore it.

In the documentation it says it’s possible.

"Starting with version 1.8.0, you can save oplog slices without the mandatory base backup snapshot. "

Attempt :

root@DESKTOP-L0E2A77:~# pbm list  --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
Backup snapshots:
  2022-12-21T08:23:37Z <physical> [restore_to_time: 2022-12-21T08:23:38Z]
  2023-01-04T07:18:21Z <physical> [restore_to_time: 2023-01-04T07:18:24Z]
  2023-01-04T07:25:41Z <physical> [restore_to_time: 2023-01-04T07:25:44Z]
pbm config --set pitr.enabled=true  --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
[pitr.enabled=true]

Logs :

2023-01-04T12:56:00.000+0530 D [pitr] start_catchup
2023-01-04T12:56:00.000+0530 E [pitr] init: catchup: no backup found. full backup is required to start PITR
2023-01-04T12:56:30.000+0530 D [pitr] start_catchup
2023-01-04T12:56:30.000+0530 E [pitr] init: catchup: no backup found. full backup is required to start PITR
2023-01-04T12:57:00.000+0530 D [pitr] start_catchup
2023-01-04T12:57:00.000+0530 E [pitr] init: catchup: no backup found. full backup is required to start PITR
2023-01-04T12:57:30.000+0530 D [pitr] start_catchup
2023-01-04T12:57:30.000+0530 E [pitr] init: catchup: no backup found. full backup is required to start PITR

Can you guide me how to perform both scenerio?

For Knowledge, I am able to take backup of Oplog over Logical backup.

Thank you in Advance.

1 Like

Hi Jyoti,

Yes, it is possible to execute oplog slices backups in both methods.

With regards to the message that you are experiencing. Can you try to use the following option:

pitr:
   oplogOnly: true

It is expected to perform the oplog backup. If it does not work, please provide the MongoDB version you are using so we can test it.

Best,

1 Like

Thank you Vini for prompt response.

I was able to get backup oplog only using :

vi + /etc/pbm_config.yaml
#added below lines
pitr:
   oplogOnly: true

But I guess It’s not listed in PBM CLi List command :

root@DESKTOP-L0E2A77:~# pbm list  --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
Backup snapshots:
  2023-01-06T06:43:52Z <physical> [restore_to_time: 2023-01-06T06:43:54Z]

PITR <on>:
root@DESKTOP-L0E2A77:~#

Logs from PBM-AGENT :


2023-01-06T12:07:27.000+0530 D [pitr] start_catchup [oplog only]
2023-01-06T12:07:27.000+0530 D [pitr] lastTS set to {1672987047 3} 2023-01-06T06:37:27
2023-01-06T12:07:27.000+0530 I [pitr] streaming started from 2023-01-06 06:37:27 +0000 UTC / 1672987047
2023-01-06T12:07:27.000+0530 D [pitr] start_ok

2023-01-06T12:13:52.000+0530 I got command backup [name: 2023-01-06T06:43:52Z, compression: s2 (level: default)] <ts: 1672987432>
2023-01-06T12:13:52.000+0530 I got epoch {1672987037 1}
2023-01-06T12:13:52.000+0530 I [pitr] got wake_up signal
2023-01-06T12:13:52.000+0530 D [backup/2023-01-06T06:43:52Z] init backup meta
2023-01-06T12:13:52.000+0530 I [pitr] wake_up for bcp 63b7c3286603af040376caee
2023-01-06T12:13:52.000+0530 D [backup/2023-01-06T06:43:52Z] nomination list for singlereplica: [[127.0.0.1:27017]]
2023-01-06T12:13:52.000+0530 D [backup/2023-01-06T06:43:52Z] nomination singlereplica, set candidates [127.0.0.1:27017]
2023-01-06T12:13:53.000+0530 I [backup/2023-01-06T06:43:52Z] backup started
2023-01-06T12:13:54.000+0530 D [backup/2023-01-06T06:43:52Z] backup cursor id: 06eeb136-3293-41ae-8f80-e440242d5c17
2023-01-06T12:13:55.000+0530 I [pitr] created chunk 2023-01-06T06:37:27 - 2023-01-06T06:43:53
2023-01-06T12:13:55.000+0530 I [pitr] pausing/stopping with last_ts 2023-01-06 06:43:53 +0000 UTC
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] set journal up to {1672987434 1}
2023-01-06T12:13:57.000+0530 I [backup/2023-01-06T06:43:52Z] uploading files
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/journal/WiredTigerLog.0000000032 100.00MB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/WiredTigerHS.wt 56.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/_mdb_catalog.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-0--3371391368073350980.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-0--4107819621118277074.wt 44.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-0-5143574458543537016.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-0-6141969711064962041.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-10--4107819621118277074.wt 20.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-12--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-14--4107819621118277074.wt 2.93MB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-15--4107819621118277074.wt 24.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-17--4107819621118277074.wt 20.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-19--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-2--3371391368073350980.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-2-5143574458543537016.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-2-5751740019876761316.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-22--4107819621118277074.wt 4.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-24--4107819621118277074.wt 4.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-26--4107819621118277074.wt 4.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-27--4107819621118277074.wt 20.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-28--4107819621118277074.wt 4.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-29--4107819621118277074.wt 4.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-35--4107819621118277074.wt 4.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-38--4107819621118277074.wt 20.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-41--4107819621118277074.wt 20.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-47--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-49--4107819621118277074.wt 164.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-5--7400535609859724419.wt 20.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-51--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-54--4107819621118277074.wt 32.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-57--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-60--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-64--4107819621118277074.wt 44.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-68--4107819621118277074.wt 36.00KB
2023-01-06T12:13:57.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-70--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/collection-8--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-1--3371391368073350980.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-1--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-1-5143574458543537016.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-1-6141969711064962041.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-11--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-13--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-16--4107819621118277074.wt 24.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-18--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-20--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-21--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-23--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-25--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-3--3371391368073350980.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-3-5143574458543537016.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-3-5751740019876761316.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-30--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-31--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-32--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-33--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-34--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-36--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-37--4107819621118277074.wt 4.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-39--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-4--3371391368073350980.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-40--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-42--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-43--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-48--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-50--4107819621118277074.wt 92.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-52--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-53--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-55--4107819621118277074.wt 32.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-56--4107819621118277074.wt 32.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-58--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-59--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-6--7400535609859724419.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-61--4107819621118277074.wt 32.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-62--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-63--4107819621118277074.wt 32.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-65--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-66--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-67--4107819621118277074.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-69--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-71--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/index-9--4107819621118277074.wt 20.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/sizeStorer.wt 36.00KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/WiredTiger.backup 142.66KB
2023-01-06T12:13:58.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/WiredTiger 50.00B
2023-01-06T12:13:59.000+0530 D [backup/2023-01-06T06:43:52Z] uploading: /var/lib/mongodb/journal/WiredTigerLog.0000000033 100.00MB
2023-01-06T12:13:59.000+0530 I [backup/2023-01-06T06:43:52Z] uploading done
2023-01-06T12:13:59.000+0530 D [backup/2023-01-06T06:43:52Z] stop cursor polling: <nil>, cursor err: <nil>
2023-01-06T12:13:59.000+0530 D [backup/2023-01-06T06:43:52Z] epoch set to {1672987439 6}
2023-01-06T12:14:01.000+0530 I [backup/2023-01-06T06:43:52Z] backup finished
2023-01-06T12:14:01.000+0530 D [backup/2023-01-06T06:43:52Z] releasing lock
2023-01-06T12:14:15.000+0530 D [pitr] start_catchup [oplog only]
2023-01-06T12:14:15.000+0530 D [pitr] lastTS set to {1672987433 4} 2023-01-06T06:43:53
2023-01-06T12:14:15.000+0530 I [pitr] streaming started from 2023-01-06 06:43:53 +0000 UTC / 1672987433
2023-01-06T12:14:15.000+0530 D [pitr] start_ok
1 Like

hey Jyoti,
If you still have issues, could you try resyncing with the storage and then check again (pbm status)?
pbm config --force-resync --file /etc/pbm_config.yaml

Please make sure to have parameters defined in the yaml file.

1 Like

Don’t seems to be working :

root@DESKTOP-L0E2A77:~# pbm config --force-resync --file /etc/pbm_config.yaml   --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
Storage resync started
root@DESKTOP-L0E2A77:~# date
Mon Jan  9 11:31:26 IST 2023
root@DESKTOP-L0E2A77:~# pbm list  --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
Backup snapshots:
  2023-01-06T06:43:52Z <physical> [restore_to_time: 2023-01-06T06:43:54Z]

PITR <on>:
root@DESKTOP-L0E2A77:~# pbm list  --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
Backup snapshots:
  2023-01-06T06:43:52Z <physical> [restore_to_time: 2023-01-06T06:43:54Z]

PITR <on>:
root@DESKTOP-L0E2A77:~# date
Mon Jan  9 11:57:59 IST 2023
root@DESKTOP-L0E2A77:~#

Logs :


2023-01-09T11:30:00.000+0530 D [pitr] lastTS set to {1673243985 9} 2023-01-09T05:59:45
2023-01-09T11:30:00.000+0530 I [pitr] streaming started from 2023-01-09 05:59:45 +0000 UTC / 1673243985
2023-01-09T11:30:00.000+0530 D [pitr] start_ok
2023-01-09T11:30:20.000+0530 I got command resync <ts: 1673244020>
2023-01-09T11:30:20.000+0530 I got epoch {1673243993 1}
2023-01-09T11:30:20.000+0530 D [resync] get lock: another operation is running: PITR incremental backup ''
2023-01-09T11:30:20.000+0530 D [resync] lock not acquired
2023-01-09T11:40:03.000+0530 I [pitr] created chunk 2023-01-09T05:59:45 - 2023-01-09T06:10:00. Next chunk creation scheduled to begin at ~2023-01-09T11:50:00
2023-01-09T11:50:03.000+0530 I [pitr] created chunk 2023-01-09T06:10:00 - 2023-01-09T06:20:00. Next chunk creation scheduled to begin at ~2023-01-09T12:00:00
1 Like

Hey if I check with pbm status then it is showing me :

root@DESKTOP-L0E2A77:~# pbm status  --mongodb-uri="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&replicaSet=singlereplica"
Cluster:
========
singlereplica:
  - singlereplica/127.0.0.1:27017 [P]: pbm-agent v2.0.2 OK


PITR incremental backup:
========================
Status [ON]

Currently running:
==================
Snapshot backup "2023-01-09T06:32:48Z", started at 2023-01-09T06:32:49Z. Status: done. [op id: 63bbb510c308a1a167962ad1]

Backups:
========
FS  /root/backup
  Snapshots:
    2023-01-09T06:32:48Z 2.02MB <physical> [restore_to_time: 2023-01-09T06:32:51Z]
    2023-01-06T06:43:52Z 2.18MB <physical> [restore_to_time: 2023-01-06T06:43:54Z]
    2022-12-19T08:56:35Z 0.00B <physical> [ERROR: some of pbm-agents were lost during the backup] [2022-12-21T08:23:15Z]
    2022-12-15T08:27:43Z 0.00B <physical> [ERROR: some of pbm-agents were lost during the backup] [2022-12-19T08:56:36Z]
    2022-12-14T12:47:10Z 0.00B <physical> [ERROR: some of pbm-agents were lost during the backup] [2022-12-15T08:27:44Z]
    2022-12-14T12:38:32Z 0.00B <physical> [ERROR: some of pbm-agents were lost during the backup] [2022-12-14T12:47:11Z]
  PITR chunks [37.69KB]:
    2023-01-09T05:53:59Z - 2023-01-09T06:32:50Z (no base snapshot)
1 Like

@Vinodh_Krishnaswamy @vinicius.grippa , My question to you is that is there any way to find or extract oplog which are backed up using PBM.

My purpose is that if my primary node is unreachable & want to search the operation from snapshot of oplog slice & once I’ll find out the timestamp from operation. So,then can apply or recovery of that timestamp only.

1 Like

Hi Jyoti,

Sorry for the late reply. I checked my backups and I can see my oplog slices. In my example:

[root@vinicius-grippa-node1 20230124]# pwd
/nfs/local_backups/pbmPitr/rs0/20230124
[root@vinicius-grippa-node1 20230124]# ls -larth
total 595M
drwxr-xr-x. 3 mongod mongod 4.0K Jan 24 00:57 ..
-rw-r--r--. 1 mongod mongod  13K Jan 24 00:57 20230124005211-2.20230124005712-21.oplog.s2
-rw-r--r--. 1 mongod mongod  15K Jan 24 00:59 20230124005712-21.20230124005922-19.oplog.s2
-rw-r--r--. 1 mongod mongod 298M Jan 24 01:00 20230124005922-19.20230124010006-2.oplog.s2
-rw-r--r--. 1 mongod mongod  14K Jan 24 01:02 20230124010006-2.20230124010219-3.oplog.s2
-rw-r--r--. 1 mongod mongod 4.6K Jan 24 01:04 20230124010219-3.20230124010419-3.oplog.s2
-rw-r--r--. 1 mongod mongod 2.9K Jan 24 01:04 20230124010419-3.20230124010434-4.oplog.s2
-rw-r--r--. 1 mongod mongod 4.0K Jan 24 01:04 20230124010434-4.20230124010443-5.oplog.s2
-rw-r--r--. 1 mongod mongod 298M Jan 24 01:07 20230124010443-5.20230124010729-3.oplog.s2
drwxr-xr-x. 2 mongod mongod 4.0K Jan 24 01:09 .
-rw-r--r--. 1 mongod mongod 5.1K Jan 24 01:09 20230124010729-3.20230124010929-3.oplog.s2

But note the slices are compressed using s2 compression. You will need to do some manual work.

My purpose is that if my primary node is unreachable & want to search the operation from snapshot of oplog slice & once I’ll find out the timestamp from operation. So,then can apply or recovery of that timestamp only.

I’m not sure I follow you. To have a non-PRIMARY, it would mean you lost two nodes (from a 3-node replica set). In this situation you plan to apply the oplog until the desired PITR?

Did you try using oplog replay optionf from PBM?

For example:

$ pbm oplog-replay --start="2023-01-24T00:52:12" --end="2023-01-24T01:15:29"
Starting oplog replay '2023-01-24T00:52:12 - 2023-01-24T01:15:29'..

For the records,

To recover in situations where you ran a drop collection, you need to recover the timestamp from the logs and apply it to the oplog-reply.

For example:

rs0:PRIMARY> db.date.drop()
true
2023-01-24T19:53:18.054+0000 I COMMAND [conn512] CMD: drop percona.date
2023-01-24T19:53:18.055+0000 I STORAGE [conn512] dropCollection: percona.date (41c4ee09-a44a-43d3-9d6e-88611b6ab4ba) - storage engine will take ownership of drop-pending collection with optime { ts: Timestamp(0, 0), t: -1 } and commit timestamp Timestamp(0, 0)
2023-01-24T19:53:18.055+0000 I STORAGE [conn512] Finishing collection drop for percona.date (41c4ee09-a44a-43d3-9d6e-88611b6ab4ba).

You need to use the timestamp from the drop command.