Error during restore "get metadata: parse physical restore status: read ts from rs.csrs/node.eis-mongo-test03:26001.hb: strconv.ParseInt: parsing "": invalid syntax"

MongoDB Percona Backup for MongoDB
,
1

We have sharded mongodb cluster with one shard:
4 config servers (2 community and 2 Percona Server for MongoDB)
4 nodes with data (2 community and 2 Percona Server for MongoDB)
Major version is 7.

I tried to restore physical backup, but got an error:

pbm --mongodb-uri “mongodb://pbmuser:pbmuser@eis-mongo-test01:26001” status
Cluster:

csrs:

  • csrs/eis-mongo-test01:26001 [P]: pbm-agent v2.5.0 OK
  • csrs/eis-mongo-test02:26001 [S]: pbm-agent v2.5.0 OK
  • csrs/eis-mongo-test03:26001 [S]: pbm-agent v2.5.0 OK
  • csrs/eis-mongo-test04:26001 [S]: pbm-agent v2.5.0 OK
    repl-set1:
  • repl-set1/eis-mongo-test05:27011 [P]: pbm-agent v2.5.0 OK
  • repl-set1/eis-mongo-test06:27011 [S]: pbm-agent v2.5.0 OK
  • repl-set1/eis-mongo-test07:27011 [S]: pbm-agent v2.5.0 OK
  • repl-set1/eis-mongo-test08:27011 [S]: pbm-agent v2.5.0 OK

PITR incremental backup:

Status [OFF]

Currently running:

(none)

Backups:

FS /u01/backups
Snapshots:
2026-03-04T13:10:40Z 103.77MB [restore_to_time: 2026-03-04T13:10:42Z]
PITR chunks [1.16MB]:
2026-03-04T13:10:43Z - 2026-03-04T13:26:50Z

pbm --mongodb-uri “mongodb://pbmuser:pbmuser@eis-mongo-test01:26001” restore 2026-03-04T13:10:40Z
Starting restore 2026-03-04T13:43:10.531378882Z from ‘2026-03-04T13:10:40Z’.Error: get metadata: parse physical restore status: read ts from rs.csrs/node.eis-mongo-test03:26001.hb: strconv.ParseInt: parsing “”: invalid syntax

Pbm logs in each node don’t show any problems and look like this:
мар 04 14:57:02 eis-mongo-test02 pbm-agent[1604190]: 2026-03-04T14:57:02.000+0300 D [restore/2026-03-04T11:53:09.527876037Z] rm tmp conf
мар 04 14:57:02 eis-mongo-test02 pbm-agent[1604190]: 2026-03-04T14:57:02.000+0300 D [restore/2026-03-04T11:53:09.527876037Z] rm tmp logs
мар 04 14:57:02 eis-mongo-test02 pbm-agent[1604190]: 2026-03-04T14:57:02.000+0300 I [restore/2026-03-04T11:53:09.527876037Z] recovery successfully finished
мар 04 14:57:02 eis-mongo-test02 pbm-agent[1604190]: 2026-03-04T14:57:02.000+0300 I change stream was closed
мар 04 14:57:02 eis-mongo-test02 pbm-agent[1604190]: 2026-03-04T14:57:02.000+0300 D [restore/2026-03-04T11:53:09.527876037Z] hearbeats stopped
мар 04 14:57:02 eis-mongo-test02 pbm-agent[1604190]: 2026/03/04 14:57:02 Exit:

After this restore attempt all mongod processes were stopped, and pbm status looked like this:
Backups:

FS /u01/backups
Snapshots:
2026-03-04T13:10:40Z 0.00B [ERROR: Backup stuck at starting stage, last beat ts: 1772629841] [2026-03-04T13:10:40Z]

How can I fix this problem?

2

Hi @Magzhanova,

The strconv.ParseInt: parsing "": invalid syntax error is a known race condition in PBM’s physical restore coordination. It’s tracked as PBM-1238.

During physical restore, each PBM agent writes a heartbeat (.hb) file to signal progress. The write is not atomic: it truncates the file first, then writes the new timestamp. If another agent reads in between those two steps, it sees an empty string and strconv.ParseInt fails. You’re running PBM 2.5.0, which has this bug. PBM 2.10.0 fixed the primary race, and PBM 2.13.0 fixed a second related race in checkHB(). Upgrading to 2.13.0+ should eliminate this error.

One concern about your topology: you’re running a mix of community MongoDB and Percona Server for MongoDB (PSMDB) nodes. Physical backup/restore relies on $backupCursor, which is a PSMDB-specific extension. PBM can route physical backups to PSMDB nodes in mixed clusters, but physical restore runs on every node. Community nodes lack the backup cursor support, so physical restore on a mixed cluster is not a tested configuration. This could cause issues beyond the heartbeat race. For physical restore to work reliably, all nodes should be running PSMDB.

If migrating all nodes to PSMDB is not feasible right now, take logical backups (pbm backup --type=logical) alongside your physical ones. Logical backup and restore use mongodump/mongorestore under the hood and work with community MongoDB. You cannot restore a physical backup using logical restore (PBM uses the backup type to determine the restore method), so you need the logical backups available as an alternative path.