Possible bug in the operator using pmm

MongoDB Percona Operator for MongoDB
1

I have setup Percona mongodb operator with enabled pmm. Everything seems fine.

NAME                                                   READY   STATUS        RESTARTS      AGE
pod/cwmdb-cfg-0                                        3/3     Running       0             9m54s
pod/cwmdb-cfg-1                                        3/3     Running       0             29s
pod/cwmdb-mongos-758749789-nknsd                       2/2     Running       0             9m19s
pod/cwmdb-mongos-758749789-qscmd                       2/2     Running       0             9m19s
pod/cwmdb-rs0-0                                        3/3     Running       0             9m54s
pod/cwmdb-rs0-1                                        3/3     Running       0             8m48s
pod/cwmdb-rs0-arbiter-0                                1/1     Running       0             9m54s
pod/monitoring-server-8489dd4889-2gbl8                 1/1     Running       2 (24h ago)   5d2h
pod/percona-server-mongodb-operator-5dd88ff7f7-f7mv7   1/1     Running       3 (24h ago)   6d4h

But when I change pmm.enabled=false and apply the changes kubectl apply -f cr.yaml only one of the pods gets applied, and the mongos ready state keeps cycling 1/2 2/2

pod/cwmdb-cfg-0                                        3/3     Running   0             11m
pod/cwmdb-cfg-1                                        2/2     Running   0             48s
pod/cwmdb-mongos-758749789-nknsd                       1/2     Running   0             11m
pod/cwmdb-mongos-758749789-qscmd                       1/2     Running   0             11m
pod/cwmdb-rs0-0                                        3/3     Running   0             11m
pod/cwmdb-rs0-1                                        3/3     Running   0             114s
pod/cwmdb-rs0-arbiter-0                                1/1     Running   0             11m
pod/monitoring-server-8489dd4889-2gbl8                 1/1     Running   2 (24h ago)   5d2h
pod/percona-server-mongodb-operator-5dd88ff7f7-f7mv7   1/1     Running   3 (24h ago)   6d4h

Any experience on this ?

1 Like
2

Hello @Darko ,

I cannot reproduce this.

I had this with PMM enabled:

my-cluster-name-cfg-0                              3/3     Running   0          29m
my-cluster-name-cfg-1                              3/3     Running   0          28m
my-cluster-name-cfg-2                              3/3     Running   0          28m
my-cluster-name-mongos-7f7846d95-dd9wb             2/2     Running   0          28m
my-cluster-name-mongos-7f7846d95-dfwvb             2/2     Running   0          28m
my-cluster-name-mongos-7f7846d95-dpsq9             2/2     Running   0          28m
my-cluster-name-rs0-0                              3/3     Running   0          29m
my-cluster-name-rs0-1                              3/3     Running   0          28m
my-cluster-name-rs0-2                              3/3     Running   0          28m
percona-server-mongodb-operator-5dd88ff7f7-47zzn   1/1     Running   0          36m

Disabled pmm and applied the CR and got to this state:

NAME                                               READY   STATUS    RESTARTS   AGE
my-cluster-name-cfg-0                              2/2     Running   0          3m59s
my-cluster-name-cfg-1                              2/2     Running   0          3m29s
my-cluster-name-cfg-2                              2/2     Running   0          4m29s
my-cluster-name-mongos-7d5659b9f9-ccnsr            1/1     Running   0          3m58s
my-cluster-name-mongos-7d5659b9f9-czvdw            1/1     Running   0          4m16s
my-cluster-name-mongos-7d5659b9f9-fshkw            1/1     Running   0          4m35s
my-cluster-name-rs0-0                              2/2     Running   0          2m39s
my-cluster-name-rs0-1                              2/2     Running   0          119s
my-cluster-name-rs0-2                              2/2     Running   0          3m10s
percona-server-mongodb-operator-5dd88ff7f7-47zzn   1/1     Running   0          40m

Is there anything specific about your deployment? Could you please share the cr.yaml ?