Bitnami image issue with mongodb-exporter:0.32.0

MongoDB Percona Operator for MongoDB
1

Description:

Looks like our version of operator installed MongoDB has been caught by the Bitnami changes of Sept 29th

Steps to Reproduce:

Restart MongoDB on a Kubernetes node

Version:

mongodb-exporter:0.32.0

Logs:

root@node1:~# kubectl describe pod deai-mongodb-rs0-1 -n mongodb
Name:         deai-mongodb-rs0-1
Namespace:    mongodb
Priority:     0
Node:         node4/192.168.52.210
Start Time:   Tue, 30 Sep 2025 10:16:32 +0200
Labels:       app.kubernetes.io/component=mongod
              app.kubernetes.io/instance=deai-mongodb
              app.kubernetes.io/managed-by=percona-server-mongodb-operator
              app.kubernetes.io/name=percona-server-mongodb
              app.kubernetes.io/part-of=percona-server-mongodb
              app.kubernetes.io/replset=rs0
              controller-revision-hash=deai-mongodb-rs0-54c4bcd96
              statefulset.kubernetes.io/pod-name=deai-mongodb-rs0-1
Annotations:  cni.projectcalico.org/containerID: fdb53d411c6418d69d995d44d9fc8eb0177872725efc74d2c4017b1122551d07
              cni.projectcalico.org/podIP: 10.233.105.24/32
              cni.projectcalico.org/podIPs: 10.233.105.24/32
              percona.com/ssl-hash: c9791b2fe1260d3efa5225d249a491c2
              percona.com/ssl-internal-hash: e8ce03c545a49d9119aeb343a92c1faa
Status:       Pending
IP:           10.233.105.24
IPs:
  IP:           10.233.105.24
Controlled By:  StatefulSet/deai-mongodb-rs0
Init Containers:
  mongo-init:
    Container ID:  containerd://a075dede1d42cc2b981f1e72a587cf81d571ba779d0edeba04d2d0fe12bf0a16
    Image:         perconalab/percona-server-mongodb-operator:1.12.0
    Image ID:      docker.io/perconalab/percona-server-mongodb-operator@sha256:e9ed11994cef3f7ab33e126484d5d5991cccc00b54d066183d1c7abe8e29b802
    Port:          <none>
    Host Port:     <none>
    Command:
      /init-entrypoint.sh
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 30 Sep 2025 10:16:52 +0200
      Finished:     Tue, 30 Sep 2025 10:16:52 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  10G
    Requests:
      memory:     6G
    Environment:  <none>
    Mounts:
      /data/db from mongod-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8djhd (ro)
Containers:
  mongod:
    Container ID:  containerd://3f14f49d08ff0537935a83a73285e0886031e42423a0b9c9c58f8591afeae85e
    Image:         percona/percona-server-mongodb:5.0.7-6
    Image ID:      docker.io/percona/percona-server-mongodb@sha256:3f4849a17236c3849a513f46caa39fbc6da0414f98d27e080fbe0496fa9e86a2
    Port:          27017/TCP
    Host Port:     0/TCP
    Command:
      /data/db/ps-entry.sh
    Args:
      --bind_ip_all
      --auth
      --dbpath=/data/db
      --port=27017
      --replSet=rs0
      --storageEngine=wiredTiger
      --relaxPermChecks
      --sslAllowInvalidCertificates
      --clusterAuthMode=x509
      --shardsvr
      --enableEncryption
      --encryptionKeyFile=/etc/mongodb-encryption/encryption-key
      --wiredTigerCacheSizeGB=4.16
      --wiredTigerIndexPrefixCompression=true
    State:          Running
      Started:      Tue, 30 Sep 2025 10:17:08 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      memory:  10G
    Requests:
      memory:   6G
    Liveness:   exec [/data/db/mongodb-healthcheck k8s liveness --ssl --sslInsecure --sslCAFile /etc/mongodb-ssl/ca.crt --sslPEMKeyFile /tmp/tls.pem --startupDelaySeconds 7200] delay=60s timeout=10s period=30s #success=1 #failure=4
    Readiness:  tcp-socket :27017 delay=10s timeout=2s period=3s #success=1 #failure=8
    Environment Variables from:
      internal-deai-mongodb-users  Secret  Optional: false
    Environment:
      SERVICE_NAME:     deai-mongodb
      NAMESPACE:        mongodb
      MONGODB_PORT:     27017
      MONGODB_REPLSET:  rs0
    Mounts:
      /data/db from mongod-data (rw)
      /etc/mongodb-encryption from deai-mongodb-mongodb-encryption-key (ro)
      /etc/mongodb-secrets from deai-mongodb-mongodb-keyfile (ro)
      /etc/mongodb-ssl from ssl (ro)
      /etc/mongodb-ssl-internal from ssl-internal (ro)
      /etc/users-secret from users-secret-file (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8djhd (ro)
  mongodb-exporter:
    Container ID:
    Image:         bitnami/mongodb-exporter:0.32.0
    Image ID:
    Port:          9216/TCP
    Host Port:     0/TCP
    Args:
      --web.listen-address=$(POD_IP):9216
      --log.level=info
      --mongodb.global-conn-pool
      --mongodb.direct-connect
      --collect-all
      --discovering-mode
      --compatible-mode
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:
      EXPORTER_USER:  <set to the key 'MONGODB_CLUSTER_MONITOR_USER' in secret 'deai-mongodb-users'>      Optional: false
      EXPORTER_PASS:  <set to the key 'MONGODB_CLUSTER_MONITOR_PASSWORD' in secret 'deai-mongodb-users'>  Optional: false
      POD_IP:          (v1:status.podIP)
      MONGODB_URI:    mongodb://$(EXPORTER_USER):$(EXPORTER_PASS)@$(POD_IP):27017
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8djhd (ro)
  backup-agent:
    Container ID:   containerd://f0279f313aac7d17a52be37afe6ce3d50d51f6211983fde8694b55970b941f1f
    Image:          percona/percona-backup-mongodb:1.7.0
    Image ID:       docker.io/percona/percona-backup-mongodb@sha256:01388bcc310de0190cad0c4cfbe8cf8edc07c5c7ea057edec137e8f58931696b
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 30 Sep 2025 10:17:14 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  1G
    Requests:
      cpu:     1
      memory:  1G
    Environment:
      PBM_AGENT_MONGODB_USERNAME:  <set to the key 'MONGODB_BACKUP_USER' in secret 'internal-deai-mongodb-users'>      Optional: false
      PBM_AGENT_MONGODB_PASSWORD:  <set to the key 'MONGODB_BACKUP_PASSWORD' in secret 'internal-deai-mongodb-users'>  Optional: false
      PBM_MONGODB_REPLSET:         rs0
      PBM_MONGODB_PORT:            27017
      SHARDED:                     TRUE
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8djhd (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  mongod-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mongod-data-deai-mongodb-rs0-1
    ReadOnly:   false
  deai-mongodb-mongodb-keyfile:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  deai-mongodb-mongodb-keyfile
    Optional:    false
  deai-mongodb-mongodb-encryption-key:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  deai-mongodb-mongodb-encryption-key
    Optional:    false
  ssl:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  deai-mongodb-ssl
    Optional:    false
  ssl-internal:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  deai-mongodb-ssl-internal
    Optional:    true
  users-secret-file:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  internal-deai-mongodb-users
    Optional:    false
  kube-api-access-8djhd:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason   Age                    From     Message
  ----    ------   ----                   ----     -------
  Normal  BackOff  3m14s (x304 over 72m)  kubelet  Back-off pulling image "bitnami/mongodb-exporter:0.32.0"

Expected Result:

Pod starts properly

Actual Result:

mongodb deai-mongodb-cfg-1 2/2 Running 0 79m 10.233.105.122 node4
mongodb deai-mongodb-rs0-1 2/3 ImagePullBackOff 0 79m 10.233.105.24 node4

Additional Information:

Not sure why this is calling a Bitnami image, as it’s available from yourselves. I’m assuming that, given the right steer, there is a file that can be edited to switch the Bitnami reference to Percona, but it would speed things up if someone would be kind enough to give me a hint where to find it.

Kind regards,

Mike

2

I’m not seeing in any CR/CRDs a mention of bitnami/mongodb-exporter image in Percona Operator for MongoDB. Isn’t it something you have added? Can you please attach the CR you deployed?

3

Hi Radoslaw,

That’s because, I believe, the issue is with the Percona MongoDB Exporter, which I assume has added a sidecar to the pod, and is now preventing startup. So although I edited one location from bitnami to percona, a second image check still showed a reference to docker,io/bitnami, which I’m trying to track down.

4

ok still… what’s the source of that CR.yaml that you deployed?

5

To be honest, I really don’t know. It’s an old deployment, and I would normally use the operator to install, but this probably predates that. Checking the statefulset, it’s correctly defined.

root@node1:~# kubectl get statefulset -n mongodb deai-mongodb-rs0 -o yaml |grep image
image: percona/percona-server-mongodb:5.0.7-6
imagePullPolicy: Always
image: percona/mongodb-exporter:0.32.0
imagePullPolicy: IfNotPresent
image: percona/percona-backup-mongodb:1.7.0
imagePullPolicy: Always
image: perconalab/percona-server-mongodb-operator:1.12.0
imagePullPolicy: Always

I’m at a bit of loss as to where to look next.