Unable to take incremental/physical backup using pbm

MongoDB Percona Backup for MongoDB
1

Hi,

I’m testing backups using the PBM agent. Logical backups work without any issues, but I’m unable to perform physical or incremental backups.

I’ve read that to use these types of backups, I need to use PSMDB. I can confirm that I’m using this distribution in version 8.0.4.

For example, when trying to perform an incremental backup, I get the following output:

bash-5.1$ pbm backup --wait --type incremental --base
Starting backup '2025-01-27T08:55:02Z'....
Waiting for '2025-01-27T08:55:02Z' backup... error
Error: upload data files: upload file `/mongodb/data/db/journal/WiredTigerLog.0000000033`: get file stat: stat /mongodb/data/db/journal/WiredTigerLog.0000000033: no such file or directory

The logs look like this:

2025-01-27T08:55:03.000+0000 I got command backup [name: 2025-01-27T08:55:02Z, compression: s2 (level: default)] <ts: 1737968102>, opid: 679749e60ea13dc92517970b
2025-01-27T08:55:03.000+0000 I got epoch {1737967784 8}
2025-01-27T08:55:03.000+0000 D [backup/2025-01-27T08:55:02Z] init backup meta
2025-01-27T08:55:03.000+0000 D [backup/2025-01-27T08:55:02Z] nomination list for rs0: [[mongodb-primary:27017]]
2025-01-27T08:55:03.000+0000 D [backup/2025-01-27T08:55:02Z] nomination rs0, set candidates [mongodb-primary:27017]
2025-01-27T08:55:04.000+0000 I [backup/2025-01-27T08:55:02Z] backup started
2025-01-27T08:55:05.000+0000 D [backup/2025-01-27T08:55:02Z] flush incremental backup history
2025-01-27T08:55:05.000+0000 D [backup/2025-01-27T08:55:02Z] backup cursor id: 2b430e29-b2dd-4376-91ec-319768a02790
2025-01-27T08:55:07.000+0000 D [backup/2025-01-27T08:55:02Z] set journal up to {1737968105 2}
2025-01-27T08:55:07.000+0000 I [backup/2025-01-27T08:55:02Z] uploading data
2025-01-27T08:55:07.000+0000 I [backup/2025-01-27T08:55:02Z] mark RS as error `upload data files: upload file `/mongodb/data/db/journal/WiredTigerLog.0000000033`: get file stat: stat /mongodb/data/db/journal/WiredTigerLog.0000000033: no such file or directory`: <nil>
2025-01-27T08:55:07.000+0000 I [backup/2025-01-27T08:55:02Z] mark backup as error `upload data files: upload file `/mongodb/data/db/journal/WiredTigerLog.0000000033`: get file stat: stat /mongodb/data/db/journal/WiredTigerLog.0000000033: no such file or directory`: <nil>
2025-01-27T08:55:07.000+0000 E [backup/2025-01-27T08:55:02Z] backup: upload data files: upload file `/mongodb/data/db/journal/WiredTigerLog.0000000033`: get file stat: stat /mongodb/data/db/journal/WiredTigerLog.0000000033: no such file or directory
2025-01-27T08:55:07.000+0000 D [backup/2025-01-27T08:55:02Z] releasing lock
2025-01-27T08:55:07.000+0000 D [backup/2025-01-27T08:55:02Z] stop cursor polling: <nil>, cursor err: <nil>

I inspected the database’s filesystem and can confirm that the file existed before starting the backup, but after PBM reported the error, the file name changed from 33 to 35.

Currently, I have a single instance of PSMDB with an initialized replica set.

I can confirm that I created the required role and user according to the documentation.

Here is the connection string I’m using:

mongodb://pbmuser:secretpwd@mongodb:27017/?authSource=admin

My config file

storage:
  type: filesystem
  filesystem:
    path: /pbm/backup
pitr:
  enabled: true
  oplogSpanMin: 10
  compression: s2
backup:
  oplogSpanMin: 0
  compression: s2
restore:
  batchSize: 500
  numInsertionWorkers: 10
  mongodLocation: /opt/pbm/bin/mongod

Running in container environment, pbm-agent and mongo are in separated containers

2

Hi, for physical backups, pbm container needs to access the data volume and also to run under the same user id as mongod. Please see Run in Docker - Percona Backup for MongoDB for more details

3

Hi, thanks for your answer

Please be informed that these processes run with the same uid

psmdb

bash-5.1$ id
uid=1001(1001) gid=0(root) groups=0(root),1001(1001)
bash-5.1$ whoami
1001
bash-5.1$ ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
1001           1  1.1  0.7 4061508 205272 ?      SLsl 13:43   0:01 /opt/mongodb/bin/mongod --config=/opt/mongodb/conf/mongodb.conf

pbm

bash-5.1$ id
uid=1001(1001) gid=0(root) groups=0(root),1001(1001)
bash-5.1$ whoami
1001
bash-5.1$ ps aux
bash: ps: command not found
bash-5.1$ top
bash: top: command not found
bash-5.1$ cat /proc/1/status | grep Uid
Uid:    1001    1001    1001    1001
4

does PBM container also have read-write access to MongoDB data volume ?

5

Shoud I mount mongodb data volume right into pbm container or the read write operations are going to be executed via network?

6

it should be mounted

1 Like
7

Jesus Christ, I thought these operation are gonna be established via network, instead of local file system.

Thank you so much for your assistance

Mounting data volume in the same path as it is on mongodb container to the pbm container resolved an issue