PBM agent on secondary dumps from primary

Hi!

We’re trying to set up PBM for a sharded mongo cluster. There’s an agent running on every mongoc and mongod node, and the backup starts just fine. On each shard, the backup gets performed by an agent running on a secondary, but the agent initiates the “mongodump” connection against the shard primary. I’ve confirmed this via db.currentOp({$all: true}) on the primary. The result is that the uplink of the primary is saturated as data is streamed from there to the secondary, and from there to the configured remote storage. This is something that certainly blocks us from taking the tool into use.

This issue was referred to in https://jira.percona.com/browse/PBM-407?focusedCommentId=247887&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-247887but I didn’t find any continuation to that particular problem. Am I doing something wrong or is there a bug?

We’re running Mongo 4.2 and PBM 1.3.2. We use TLS with certificate authentication and the PBM_MONGODB_URI is of the form:

mongodb://pbmuser:pass@local.node.fqdn:27017/?tls=true&tlsCertificateKeyFile=/path/to/certkey&tlsCAFile=/path/to/cafile

Hi, it is possible there is an incorrect connection string being passed somewhere in the code. Would you mind opening a bug in jira.percona.com so we can get some dev eyes on this?

I added debug logging to right before the MongoDump is initiated, and the connection string is the exact same one as configured. I think the problem is that the connection of the MongoDump isn’t initialized with the “direct” connection option, which leads to the driver autodiscovering the primary and connecting to it instead of the local node. I made a pull request on this: https://github.com/percona/percona-backup-mongodb/pull/446 . I’ve tested that it works with our setup. Should I still open a jira issue as well?

Thanks for the PR! I will have somebody check this asap.

Pull request https://github.com/percona/percona-backup-mongodb/pull/446 looks good to go.