Hi,
I’m facing strange issue
I created two containers in my Docker environment - Percona Server for MongoDB and Percona Backup for MongoDB. The second one includes pbm-agent.
I’m unable to connect from Percona Backup to Percona Server in order to take a backup. Percona Backup always tries to reach localhost. The container’s IP that contains Percona Server is 10.89.3.18
Please see error below
Error: get status of cluster: connect to `rs0` [rs0/localhost:27017]: ping: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: localhost:27017, Type: Unknown, Last error: dial tcp 127.0.0.1:27017: connect: connection refused }, ] }
Connection string:
mongodb://pbmuser:secretpwd@10.89.3.18:27017/?authSource=admin&replicaSet=rs0&directConnection=true
I decided to try to cheat a container with replacing the localhost address and swapping it to 10.89.3.18 in /etc/hosts/
It worked out
Is there any possibility to avoid redirecting my IP to localhost?
Thanks in advance
@Iliterallyneedhelp,
Did you follow the documentation?
https://hub.docker.com/r/percona/percona-backup-mongodb
There is a parameter to use when launching the PBM container to specify the URI of your mongo server.
Hi,
Thanks for your response. Please be informed I’ve added the connection string in my Dockerfile so I thought it’s not necessary at this point. Is it?
Both containers are running in the same Docker Network. They’re being run with the following commands:
docker run -p 27017:27017 --network perconanet -d percona-server:1.2
docker run --network perconanet -d percona-backup:0.12
Please see my PBM Dockerfile below:
FROM docker.io/percona/percona-backup-mongodb:2.3.1
EXPOSE 28017
ENV PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@10.89.3.18:27017/?authSource=admin&replicaSet=rs0&directConnection=true"
USER root
COPY pbm-config.yaml .
RUN rm /etc/sysconfig/pbm-agent \
&& echo "PBM_MONGODB_URI='$PBM_MONGODB_URI'" > /etc/sysconfig/pbm-agent \
&& chmod 640 /etc/sysconfig/pbm-agent \
&& pbm config --file pbm-config.yaml
Then for Percona Server
FROM docker.io/percona/percona-server-mongodb:7.0.2
EXPOSE 27017
#VOLUME /data
USER root
COPY script.sh .
RUN rm -f etc/mongod.conf
COPY mongod.conf /etc
RUN chmod +x script.sh
USER mongodb
CMD ["mongod", "--wiredTigerCacheSizeGB", "0.25", "--replSet", "rs0", "--bind_ip", "0.0.0.0"]
With the configuration above I’m getting such error messages:
2023-12-13T06:31:25.727+0000 I NETWORK [js] Starting new replica set monitor for rs0/10.89.3.3:27017
2023-12-13T06:31:25.728+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to 10.89.3.3:27017
2023-12-13T06:31:25.729+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for rs0 is rs0/localhost:27017
2023-12-13T06:31:25.729+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to localhost:27017
2023-12-13T06:31:26.228+0000 W NETWORK [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set rs0
2023-12-13T06:31:26.228+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Cannot reach any nodes for set rs0. Please check network connectivity and the status of the set. This has happened for 1 checks in a row.
But if I edit /etc/hosts file in the way shown below it seems to be working properly
[root@59a82c7385e2 etc]# cat hosts
10.89.3.18 localhost
With edited hosts file my logs look like this:
2023-12-13T06:33:01.840+0000 I NETWORK [js] Starting new replica set monitor for rs0/10.89.3.3:27017
2023-12-13T06:33:01.840+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to 10.89.3.3:27017
2023-12-13T06:33:01.841+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for rs0 is rs0/localhost:27017
2023-12-13T06:33:01.841+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to localhost:27017
2023-12-13T06:33:01.842+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for rs0 is rs0/localhost:27017
1
+ exit_status=0
++ grep -E '^([0-9]+)$' /tmp/tmp.oNyx38678P
+ rs_size=1
+ [[ 0 == 0 ]]
+ [[ 1 -ge 1 ]]
+ break
+ rm /tmp/tmp.oNyx38678P
+ exec pbm-agent
2023-12-13T06:33:01.000+0000 I pbm-agent:
Version: 2.3.1
Platform: linux/amd64
GitCommit: 8c4265cfb2d9a7581b782a829246d8fcb6c7d655
GitBranch: release-2.3.1
BuildTime: 2023-11-29_13:31_UTC
GoVersion: go1.19
2023-12-13T06:33:01.000+0000 I node: rs0/localhost:27017
2023-12-13T06:33:01.000+0000 I starting PITR routine
2023-12-13T06:33:01.000+0000 I listening for the commands
2023-12-13T06:33:06.000+0000 W [agentCheckup] get current storage status: query mongo: mongo: no documents in result
I tried to:
- use localhost directly in MongoDB connection string
- build new Percona Server image both with and without “–bind_ip”, “0.0.0.0”
- set connection string with Replica Set parameter only
/?replicaSet=rs0
Hi,
PBM connects to the 10.89.3.18 node, gets the list of members in the replset, and connects to the member using provided hostname from the list.
- set and use a unique hostname in the network for your MongoDB and give it to PBM
- do not set
directConnection and replicaSet params, PBM ignores them (user:pwd@host:port is enough)
Also, you can avoid building custom images for PSMDB and PBM.
Try this way:
#!/usr/bin/env bash
# get UID used by mongod, pbm-agent and pbm-cli to share filesystem permissions
MONGODB_USER=$(docker image inspect percona/percona-server-mongodb:7.0-multi --format '{{.ContainerConfig.User}}')
# prepare volumes
docker volume create psmdb-rs000-config
docker volume create psmdb-rs000-scripts
docker run --rm \
-v "$(pwd):/host-files" \
-v "psmdb-rs000-config:/cfg" \
-v "psmdb-rs000-scripts:/scripts" \
busybox sh -c 'cp /host-files/mongod.conf /cfg/ && cp /host-files/*.js /scripts/'
# create network for cluster
docker network create psmdb-cluster-net
# create mongodb "--dbpath" volume
docker volume create psmdb-rs000-data
# create mongodb log volume
docker volume create psmdb-rs000-log
docker run -d \
--restart "unless-stopped" \
--name "psmdb-rs000" \
--hostname "psmdb-rs000-host" \
--net "psmdb-cluster-net" \
-u "$MONGODB_USER" \
-p "27017:27017" \
-v "psmdb-rs000-config:/cfg" \
-v "psmdb-rs000-scripts:/scripts" \
-v "psmdb-rs000-data:/data/db" \
-v "psmdb-rs000-log:/var/log/mongo" \
percona/percona-server-mongodb:7.0-multi \
-f "/cfg/mongod.conf" \
--dbpath "/data/db" \
--logpath "/var/log/mongo/mongod.log" \
--wiredTigerCacheSizeGB="0.25" \
--replSet="rs0" \
--bind_ip_all \
--port "27017"
# wait for mongod ready
sleep 2
docker exec psmdb-rs000 mongosh --quiet --eval 'rs.initiate()'
docker exec psmdb-rs000 mongosh --quiet '/scripts/add-users.js'
# create storage for backups
docker volume create pbm-rs000-storage
docker run --rm -it -v "pbm-rs000-storage:/s" busybox chown -R "$MONGODB_USER" /s
docker run -d \
--restart "unless-stopped" \
--name "pbm-rs000" \
--net "psmdb-cluster-net" \
-u "$MONGODB_USER" \
-v "pbm-rs000-storage:/mnt/storage" \
-v "psmdb-rs000-data:/data/db" \
-e PBM_MONGODB_URI='pbmuser:secretpwd@psmdb-rs000-host:27017' \
percona/percona-backup-mongodb:2.3.1-multi
# wait for pbm ready
sleep 2
# apply config (from host fs)
docker exec -i pbm-rs000 pbm config --file='-' <./pbm-config.yaml
sleep 2
# check status
docker exec pbm-rs000 pbm status
add-users.js
db.getSiblingDB('admin').createUser({
user: 'adm',
pwd: 'pass',
roles: ['root'],
});
db.getSiblingDB('admin').createRole({
'role': 'pbmAgent',
'privileges': [{
'resource': { 'anyResource': true },
'actions': ['anyAction'],
}],
'roles': [
'backup',
'restore',
'clusterAdmin',
],
});
db.getSiblingDB('admin').createUser({
user: 'pbmuser',
pwd: 'secretpwd',
roles : ['pbmAgent'],
});
./pbm-config.yml
storage:
type: filesystem
filesystem:
path: /mnt/storage/backups
1 Like
Hi,
The incorrect value in replica set declaration was the issue indeed.
It’s working now.
Thanks for your assistance, a thread can be closed