Pbm backup error in a replicaset non-sharded scenario

MongoDB Percona Backup for MongoDB
1

I’m tring to configure percona backup for mongodb for my non-sharded replicaset built on 3 nodes. The replicaset is in production and is correctly working.
The percona configuration seems to be ok as the “pbm status” states the 3 agents are OK. If I try to start a backup I get this error:

root@mongo01-p-mi2:~# pbm status

Cluster:
========
replicaset01:
  - replicaset01/mongo01-p-mi2:27017: pbm-agent v1.5.0 OK
  - replicaset01/mongo02-p-mi2:27017: pbm-agent v1.5.0 OK
  - replicaset01/mongo03-p-mi2:27017: pbm-agent v1.5.0 OK

PITR incremental backup:
========================
Status [OFF]

Currently running:
==================
(none)

Backups:
========
FS  /pbm
  Snapshots:
    2021-08-10T07:05:09Z 0.00B [ERROR: mongodump: read data: make dump: error creating intents to dump: error creating intents for database config: error counting config.system.indexBuilds: (Unauthorized) not authorized on config to execute command { count: "system.indexBu
ilds", lsid: { id: UUID("81c5e8a5-ebb7-48a3-b44e-3f03afb4dde7") }, $clusterTime: { clusterTime: Timestamp(1628579130, 11), signature: { hash: BinData(0, 03DD5EC32F9B3AE815B55783C76D327B9373BFFC), keyId: 6963975662973485059 } }, $db: "config", $readPreference: { mode: "prim
aryPreferred" } }.] [2021-08-10T07:05:30]

I’ve followed the documentation to create the pbmuser and to install and configure the agents.
This is the user:

"_id" : "admin.pbmuser",
"userId" : UUID("03408b2e-7729-4edf-a5f1-558a531a0c42"),
"user" : "pbmuser",
"db" : "admin",
"roles" : [
{
"role" : "pbmAnyAction",
"db" : "admin"
},
{
"role" : "root",
"db" : "admin"
},
{
"role" : "restore",
"db" : "admin"
},
{
"role" : "readWrite",
"db" : "admin"
},
{
"role" : "clusterMonitor",
"db" : "admin"
},
{
"role" : "backup",
"db" : "admin"
}
],
"mechanisms" : [
"SCRAM-SHA-1",
"SCRAM-SHA-256"
]
}

I’m using this type of URI in the /etc/default/pbm-agent file:

export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@mongocsvr1:27018,mongocsvr2:27018,mongocsvr3:27018/?replicaSet=configrs"

I’ve also created this file to set the secondaryOk:

root@mongo01-p-mi2:~# more /etc/mongorc.js 
rs.secondaryOk()
root@mongo01-p-mi2:~#

When I try to run mongodump directly from command line with the pbmuser it does start correctly the dump.

I’m out of ideas, I’ve tried to add root role to user, I’ve tried with and without authsource=admin in URI.

The “pbm backup” command doesn’t give any error, but then from the “pbm status” command i have that error.

The only time it run correctly was when I stopped the 2 agents on the secondary nodes

Need some help

1 Like
2

Hi @Andrea_Odorisio

As I can you’re using replica set connection type for the agents. But pbm-agent processes should connect to their localhost mongod with a standalone type of connection. Moreover the ?replicaSet=configrs but as there is no configrs replica set in your cluster. May this confuses mongo.
Anyway, please try to connect each agent directly to it’s local host (e.g. PBM_MONGODB_URI=“mongodb://pbmuser:secretpwd@mongocsvr1:27018”, PBM_MONGODB_URI=“mongodb://pbmuser:secretpwd@mongocsvr2:27018”, "PBM_MONGODB_URI=“mongodb://pbmuser:secretpwd@mongocsvr3:27018”)

1 Like
3

Hi @Andrew_Pogrebnoi ,

The string i posted was like and example, this is the correct one I’m using on each agent:

PBM_MONGODB_URI=“mongodb://admin:admin@localhost:27017/?replicaSet=replicaset01”

But it makes no difference, I’m always getting the same error.

What I saw is that none of the configured users in the replicaset can access to that collection system.indexBuilds … it seems like there is something wrong with that collection, but it’s a system collection, i didn’t do nothing there. Any ideas?

1 Like
4

Ah, I see. I found the problem. This collection should be excluded from the backup. There is a commit in mongodump/mongorestore tools: TOOLS-2537: Ignore config.system.indexBuilds namespace (#242) · mongodb/mongo-tools@d11f1d4 · GitHub. But PBM 1.5 uses an older version of these libs.

In the upcoming PBM v1.6 (should be released in the following days) these dependencies are updated and include mentioned commit.

1 Like