Cannot take backup on sharded cluster

Cannot take backup via pbm on sharded mongodb cluster, works ok with single-node mongodb.
After trying to run pbm backup the following error occurred: Failed with “couldn’t get response from all shards: convergeClusterWithTimeout: reached converge timeout”. Mongodb uri is set to config set replicaset like ‘mongodb://pbmuser:pbmpass@cs1:27019,cs2:27019,cs3:27019/?authSource=admin&replicaSet=csReplSet’. Credentials are valid, pbm-agent is present on all mongod nodes accross all shards + config set nodes.
In logs I see messages like:Jun  4 08:46:43 pbm-agent[15215]: 2020/06/04 08:46:43 Got command resyncBcpListJun  4 08:46:43 pbm-agent[15215]: 2020/06/04 08:46:43 [INFO] resync_list: operation has been scheduled on another replset nodeJun  4 08:47:48 pbm-agent[15215]: 2020/06/04 08:47:48 Got command backup 2020-06-04T08:47:47ZJun  4 08:47:48 pbm-agent[15215]: 2020/06/04 08:47:48 Backup has been scheduled on another replset nodeJun  5 03:30:01 pbm-agent[15215]: 2020/06/05 03:30:01 Got command backup 2020-06-05T03:30:01ZJun  5 03:30:01 pbm-agent[15215]: 2020/06/05 03:30:01 Backup has been scheduled on another replset node
Jun  5 08:00:36 pbm-agent[18799]: 2020/06/05 08:00:36 Got command backup 2020-06-05T08:00:36ZJun  5 08:00:36 pbm-agent[18799]: 2020/06/05 08:00:36 Backup 2020-06-05T08:00:36Z started on node csReplSet/10.1.1.33:27019Jun  5 08:00:52 pbm-agent[18799]: 2020/06/05 08:00:52 Mark backup as failed couldn't get response from all shards: convergeClusterWithTimeout: reached converge timeout: <nil>Jun  5 08:00:52 pbm-agent[18799]: 2020/06/05 08:00:52 [ERROR] backup: couldn’t get response from all shards: convergeClusterWithTimeout: reached converge timeout
How can I enable debug logs to see more? Or maybe you know the reason?
MongoDB version - 4.0.6PBM version - 1.1.3
Thanks in advance.

Hi @vvol

Each pbm-agent processes should connect to their localhost mongod with a standalone type of connection. So agent’s URIs should be in format “mongodb://pbmuser:pbmpass@cs1:27019”, “mongodb://pbmuser:pbmpass@cs2:27019”, “mongodb://pbmuser:pbmpass@rs0:27019” and so on.  Since each agent should serve only one node.

Now we have a bit confusing example in the documentation. I think we will fix it.

Cheers!

Thanks for reply! Backup was successful after I had changed URI.