MongoDB loadbalancers svc stuck in pending state, without external IP address

1

kubectl get svc -n obto
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mongodb-zjc-rs0 ClusterIP None 27017/TCP 177m
mongodb-zjc-rs0-0 LoadBalancer 10.43.43.170 10.0.0.221,10.0.0.222,10.0.0.63,10.0.0.64,10.0.0.65,10.0.0.66 27017:31502/TCP 177m
mongodb-zjc-rs0-1 LoadBalancer 10.43.78.19 27017:31922/TCP 120m
mongodb-zjc-rs0-2 LoadBalancer 10.43.249.115 27017:30536/TCP 103m
percona-xtradb-cluster-operator ClusterIP 10.43.50.190

kubectl get pods -n obto
NAME READY STATUS RESTARTS AGE
mongodb-zjc-rs0-0 2/2 Running 5 (165m ago) 3h
mongodb-zjc-rs0-1 2/2 Running 5 (108m ago) 123m
mongodb-zjc-rs0-2 2/2 Running 9 (91m ago) 123m
percona-postgresql-operator-55b4fb5ff7-r7h68 1/1 Running 0 169m
percona-server-mongodb-operator-689595cd68-cjbpt 1/1 Running 0 6h46m
percona-xtradb-cluster-operator-5c47f48cb4-jt5pv 1/1 Running 1 (4h14m ago) 6h46m

kubectl get psmdb -n obto
NAME ENDPOINT STATUS AGE
mongodb-zjc initializing 3h10m

And psmdb is stuck in initializing state

2

This seems to be the problem with your network plugin (cni) that provisions load balancers for you.
Can you describe the service resource and see?

kubectl describe svc <service name>

3

kubectl --kubeconfig=k3s.yaml describe svc mongodb-zjc-rs0-1 -n obto
Name: mongodb-zjc-rs0-1
Namespace: obto
Labels: -app.kubernetes.io/component=external-service
-app.kubernetes.io/instance=mongodb-zjc
-app.kubernetes.io/managed-by=percona-server-mongodb-operator
-app.kubernetes.io/name=percona-server-mongodb
-app.kubernetes.io/part-of=percona-server-mongodb
-app.kubernetes.io/replset=rs0
Annotations: -percona.com/last-config-hash:
eyJwb3J0cyI6W3sibmFtZSI6Im1vbmdvZGIiLCJwb3J0IjoyNzAxNywidGFyZ2V0UG9ydCI6MjcwMTd9XSwic2VsZWN0b3IiOnsic3RhdGVmdWxzZXQua3ViZXJuZXRlcy5pby9wb2…
Selector: -statefulset.kubernetes.io/pod-name=mongodb-zjc-rs0-1
Type: LoadBalancer
IP: 10.43.78.19
Port: mongodb 27017/TCP
TargetPort: 27017/TCP
NodePort: mongodb 31922/TCP
Endpoints: 10.42.4.68:27017
Session Affinity: None
External Traffic Policy: Cluster
LoadBalancer Source Ranges: 0.0.0.0/0
Events:

4

Interestingly, this behavior is not 100% consistent. I was able to run my PSmdb cluster after I deleted everyting few times. Not sure how this is a problem with the CNI. I am running a mongodb replicaset using community operator and several other services, in other namespaces, without issues.

5

@Divyansh_Verma can you please book me here Zoom Scheduler and we can have a quick debugging session? Where you can also share with me your use cases.

6

I just did for the 9/10. Appreciate it. Looking forward to our discussion.