How to port forward to Primary in Kubernetes Cluster

MongoDB Percona Distribution for MongoDB
1

I want to port forward to my Primary in the ReplicaSet on my Kubernetes Cluster. This is how I am doing it now and it always connects to the secondary:

kubectl port-forward svc/percona-db-psmdb-db-rs0 -n psmdb 27017:27017

Then when I connect:

mongosh 'mongodb://user:pass@localhost:27017/ms_user?ssl=false'

Connecting to:		mongodb://<credentials>@localhost:27017/user?ssl=false&directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.5.4
Using MongoDB:		4.4.10-11
Using Mongosh:		1.5.4

Then when I want to do operations I get:

rs0 [direct: secondary] ms_user> db.users.drop()
MongoServerError: not primary
1 Like
2

Hi,

I wonder if the namespace flag you are using should be before the subcommand, like:

kubectl -n psmdb port-forward svc/percona-db-psmdb-db-rs0 27017:27017

I don’t have a test instance at hand now, so please test this and let me know if it worked. Otherwise, I can spin something up and test.

1 Like
3

Also, what do you get from kubectl -n namespace get services?

1 Like
4

Hi @anon17965173 ,
How do you install database and which services do you have setup?
I see you use helm if I’m not mistaking, this is what I did and it seems to work for me, but probably I did something different than in your setup.

Install DB:

helm install operator percona/psmdb-operator --namespace helm-test
helm install psmdb-db percona/psmdb-db --namespace helm-test --set sharding.enabled=false --set image.tag=4.4.16-16

so in my setup sharding is disabled and this is the service which is setup and to which I port-forward:

NAME           TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)     AGE
psmdb-db-rs0   ClusterIP   None         <none>        27017/TCP   7m51s

# port forwarding
$ kubectl port-forward svc/psmdb-db-rs0 -n helm-test 27017:27017
Forwarding from 127.0.0.1:27017 -> 27017
Forwarding from [::1]:27017 -> 27017

When I connect it seems to be always primary:

$ mongosh 'mongodb://<credentials>@localhost:27017/admin?ssl=false'
Current Mongosh Log ID: 63aec47a83658493db6015b4
Connecting to:          mongodb://<credentials>@localhost:27017/admin?ssl=false&directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.1
Using MongoDB:          4.4.16-16
Using Mongosh:          1.6.1

rs0 [direct: primary] admin> use test;
switched to db test

rs0 [direct: primary] test> db.korisnici.insert({ name: "Tomislav" })
{
  acknowledged: true,
  insertedIds: { '0': ObjectId("63aec48d83658493db6015b5") }
}
2 Likes
5

Hello,
It seems I have the same problem and I am not able to solve it with the methods you mentioned. Interestingly, when I create a pod in cluster and connect within it to mongodb, I connect to the primary node. When I port-forward locally, I get connected to the secondary node. Might this be related to the connection string and how mongodb resolves nodes?

How it goes locally:

$ kubectl get services -n perconamongodb
NAME                        TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)     AGE
perconamongodbcluster-rs0   ClusterIP   None         <none>        27017/TCP   176m

$ kubectl port-forward service/perconamongodbcluster-rs0 27017:27017 -n perconamongodb
Forwarding from 127.0.0.1:27017 -> 27017
Forwarding from [::1]:27017 -> 27017

$ mongosh "mongodb://user:pass@localhost:27017/helloworld?ssl=false&replicaSet=rs0&directConnection=true"
Current Mongosh Log ID: 65cb6753f23d6c23d8e058c5
Connecting to:          mongodb://<credentials>@localhost:27017/helloworld?ssl=false&replicaSet=rs0&directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+2.1.4
Using MongoDB:          6.0.9-7
Using Mongosh:          2.1.4

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

rs0 [direct: secondary] helloworld>

How it goes on the pod I have created in cluster (from the self: true you can see that the connection points to primary node):

[mongodb@percona-client /]$  mongo "mongodb://user:pass@perconamongodbcluster-rs0.perconamongodb.svc.cluster.local/helloworld?ssl=false&replicaSet=rs0&directConnection=true"
<min?ssl=false&replicaSet=rs0&directConnection=true"
Percona Server for MongoDB shell version v4.4.24-23
connecting to: mongodb://perconamongodbcluster-rs0.perconamongodb.svc.cluster.local:27017/helloworld?compressors=disabled&directConnection=true&gssapiServiceName=mongodb&replicaSet=rs0&ssl=false
Implicit session: session { "id" : UUID("05518a51-a1a5-47fd-933d-12d6804b1325") }
Percona Server for MongoDB server version: v6.0.9-7
WARNING: shell and server versions do not match

{
                        "_id" : 1,
                        "name" : "perconamongodbcluster-rs0-1.perconamongodbcluster-rs0.perconamongodb.svc.cluster.local:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 8898,
                        "optime" : {
                                "ts" : Timestamp(1707829462, 2),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2024-02-13T13:04:22Z"),
                        "lastAppliedWallTime" : ISODate("2024-02-13T13:04:22.021Z"),
                        "lastDurableWallTime" : ISODate("2024-02-13T13:04:22.021Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1707820581, 1),
                        "electionDate" : ISODate("2024-02-13T10:36:21Z"),
                        "configVersion" : 8,
                        "configTerm" : 2,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                }

What might be the reason to this? Otherwise, how to develop locally and test data insertion to the database?

I have deployed the Percona mongodb operator on Azure AKS, following this guide: Install on Microsoft Azure Kubernetes Service (AKS) - Percona Operator for MongoDB

Thanks.