Issue On PBM Backup

MongoDB
1

We have a replica set of 5 nodes, including 1 arbiter, and we are running PBM backups on the server. This morning, we encountered the following error while executing the pbm status command and while trying to apply the config file. We have already validated the issue by upgrading to the latest PBM version, but the problem persists. Please assist us in resolving this issue.

[root@N1VPCOLPOCD0006 bot_mongo]# pbm config --set=“pitr.enabled=false” --mongodb-uri=‘mongodb://pbmuser:password@10.222.234.218:27017,10.222.234.219:27017,10.242.146.32:27017,10.242.146.37:27017/?authSource=admin&replicaSet=bot_mongo’
Error: set pitr.enabled: write to db: get cluster time: no clusterTime in response

Note: All the MongoDB servers and PBM agents are running, and we are able to connect to the MongoDB shell using the PBM backup credentials.

[root@N1VPCOLPOCD0006 bot_mongo]# pbm status --mongodb-uri=‘mongodb://pbmuser:password@10.222.234.218:27017,10.222.234.219:27017,10.242.146.32:27017,10.242.146.37:27017/?authSource=admin&replicaSet=bot_mongo’
Error: get status of cluster: read cluster time: no clusterTime in response

PBM Logs Output

[root@N1VPCOLPOCD0006 bot_mongo]# pbm logs --mongodb-uri=‘mongodb://pbmuser:password@10.222.234.218:27017,10.222.234.219:27017,10.242.146.32:27017,10.242.146.37:27017/?authSource=admin&replicaSet=bot_mongo’
2024-11-03T09:11:32Z E [bot_mongo/10.242.146.37:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:32Z E [bot_mongo/10.222.234.218:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:35Z E [bot_mongo/10.222.234.219:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:37Z E [bot_mongo/10.242.146.32:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:37Z E [bot_mongo/10.242.146.37:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:37Z E [bot_mongo/10.222.234.218:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:40Z E [bot_mongo/10.222.234.219:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:40Z E [bot_mongo/10.242.146.32:27017] [pitr] init: check if already run: read cluster time: no clusterTime in response
2024-11-03T09:11:42Z E [bot_mongo/10.242.146.32:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:42Z E [bot_mongo/10.242.146.37:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:42Z E [bot_mongo/10.222.234.218:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:45Z E [bot_mongo/10.222.234.219:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:47Z E [bot_mongo/10.242.146.32:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:47Z E [bot_mongo/10.242.146.37:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:47Z E [bot_mongo/10.222.234.218:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:50Z E [bot_mongo/10.222.234.219:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:52Z E [bot_mongo/10.222.234.218:27017] [pitr] init: check if already run: read cluster time: no clusterTime in response
2024-11-03T09:11:52Z E [bot_mongo/10.242.146.32:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:52Z E [bot_mongo/10.242.146.37:27017] [agentCheckup] set status: get cluster time: no clusterTime in response
2024-11-03T09:11:52Z E [bot_mongo/10.222.234.218:27017] [agentCheckup] set status: get cluster time: no clusterTime in response

PBM Config file

pitr:
enabled: true
oplogSpanMin: 0
compression: s2
storage:
type: filesystem
filesystem:
path: /bot_mongo/Backup
backup:
oplogSpanMin: 0
compression: s2

2

Hi @_Chandrika_Pokuri
Welcome to the community!
As we can see the error, pbm pick the clusterTime from the replica set, can you please check if your replica set is configured properly and working fine.

Thanks,
Mukesh

3

Hi @Mukesh_Kumar

The cluster is working properly, but there was an issue.

Mongo version: 4.4
Installed PBM version : 2.4

4

Hi @Mukesh_Kumar

After reviewing the output of rs.status(), we noticed that the $clusterTime information is missing. We believe that adding this parameter could help resolve the issue. Is there a way to include this parameter in the rs.status() output?

5

Hi @_Chandrika_Pokuri
Regarding $clusterTime, it should come by default upon deploying replica set. Please check the documentation of deploying a MongoDB replica set.

Also, I believe, you have shared incorrect installed pbm version as till now, only released pbm v2.7.
For your reference, sharing the pbm release notes.

Now, for MongoDB v4.4 which has already reached EOL, I would recommend upgrading to latest stable version ASAP.

Thanks,
Mukesh

6

Hi @_Chandrika_Pokuri
Also, I would recommend reviewing the supported MongoDB version for pbm to work smoothly.
As stated over here,

End-of-life MongoDB versions may work with PBM, but they are not tested for compatibility. Consider using an previous PBM version in this case.

Thanks,
Mukesh

7

Hi @mukesh

It was a typo mistake,We have installed PBM version 2.4.1.

Yes, $clusterTime is included by default, but we suddenly lost this in the rs.status() output. We plan to validate this by restarting MongoDB. If you have any alternative suggestions, we would be happy to hear them.