Hi all,
I’m new to installing Percona for MongoDB. I’m deploying it on RKE2 using the official PMDB Helm chart with the Percona Operator. I’ve overridden some default values as follows:
psmdb-db:
sharding:
enabled: false
backup:
enabled: false
unsafeFlags:
replsetSize: false
replsets:
rs0:
size: 3
volumeSpec:
pvc:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
The first replica in my ReplicaSet starts successfully, but the liveness check fails with this error:
[mongodb@test-oak-mongodb-rs0-0 db]$ /opt/percona/mongodb-healthcheck k8s liveness --ssl --sslInsecure --sslCAFile /etc/mongodb-ssl/ca.crt --sslPEMKeyFile /tmp/tls.pem --startupDelaySeconds 7200
2025-10-31T11:29:47.597Z INFO Running mongodb-healthcheck {"commit": "c7a8f111326700320a918e134b8522f79e702cc1", "branch": "release-1-21-0"}
2025-10-31T11:29:47.598Z INFO Running Kubernetes liveness check for {"component": "mongod"}
2025-10-31T11:29:47.598Z DEBUG Loading SSL/TLS PEM certificate {"certificate": "/tmp/tls.pem"}
2025-10-31T11:29:47.598Z DEBUG Loading SSL/TLS Certificate Authority: %s {"ca": "/etc/mongodb-ssl/ca.crt"}
2025-10-31T11:29:47.598Z DEBUG Connecting to mongodb {"hosts": ["localhost:27017"], "ssl": true, "ssl_insecure": true}
2025-10-31T11:29:47.598Z DEBUG Enabling authentication for session {"user": "clusterMonitor"}
2025-10-31T11:29:57.625Z ERROR Failed to perform check {"error": "member failed Kubernetes liveness check: connection error: filed to dial mongo: ping mongo: connection() error occurred during connection handshake: auth error: sasl conversation error: unable to authenticate using mechanism \"SCRAM-SHA-1\": (AuthenticationFailed) Authentication failed.", "errorVerbose": "connection() error occurred during connection handshake: auth error: sasl conversation error: unable to authenticate using mechanism \"SCRAM-SHA-1\": (AuthenticationFailed) Authentication failed.\nping mongo\ngithub.com/percona/percona-server-mongodb-operator/pkg/psmdb/mongo.Dial\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/psmdb/mongo/mongo.go:123\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/db.Dial\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/db/db.go:48\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/healthcheck.HealthCheckMongodLiveness\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/healthcheck/health.go:62\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/tool.(*App).Run\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/tool/tool.go:95\nmain.main\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/main.go:45\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:285\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1693\nfiled to dial mongo\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/db.Dial\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/db/db.go:50\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/healthcheck.HealthCheckMongodLiveness\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/healthcheck/health.go:62\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/tool.(*App).Run\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/tool/tool.go:95\nmain.main\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/main.go:45\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:285\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1693\nconnection error\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/healthcheck.HealthCheckMongodLiveness\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/healthcheck/health.go:64\ngithub.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/tool.(*App).Run\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/tool/tool.go:95\nmain.main\n\t/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/main.go:45\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:285\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1693\nmember failed Kubernetes liveness check"}
main.main
/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/main.go:46
runtime.main
/usr/local/go/src/runtime/proc.go:285
2025-10-31T11:29:57.628Z ERROR failed to rotate logs {"error": "chown mongod-data/logs/mongodb-healthcheck.log: operation not permitted"}
main.main
/go/src/github.com/percona/percona-server-mongodb-operator/cmd/mongodb-healthcheck/main.go:48
runtime.main
/usr/local/go/src/runtime/proc.go:285
The second pod doesn’t start at all. I have only this log
+ exec numactl --interleave=all mongod --bind_ip_all --auth --dbpath=/data/db --port=27017 --replSet=rs0 --storageEngine=wiredTiger --relaxPermChecks --clusterAuthMode=x509 --enableEncryption --encryptionKeyFile=/etc/mongodb-encryption/encryption-key --wiredTigerCacheSizeGB=0.25 --wiredTigerIndexPrefixCompression=true --quiet --tlsMode preferTLS --sslPEMKeyFile /tmp/tls.pem --tlsAllowInvalidCertificates --tlsClusterFile /tmp/tls-internal.pem --tlsCAFile /etc/mongodb-ssl/ca.crt --tlsClusterCAFile /etc/mongodb-ssl-internal/ca.crt --logpath /data/db/logs/mongod.log --logRotate rename --logappend
{"t":{"$date":"2025-10-31T11:31:31.370Z"},"s":"W", "c":"CONTROL", "id":23321, "ctx":"main","msg":"Option: This name is deprecated. Please use the preferred name instead.","attr":{"deprecatedName":"sslPEMKeyFile","preferredName":"tlsCertificateKeyFile"}}
Has anyone seen this behavior before?
Thanks in advance!