MongoDB ReplicaSet Cluster with TLS and Letsencrypt/cert-manager

MongoDB Percona Server for MongoDB
,
1

Hello!

We are attempting to create a MongoDB ReplicaSet Cluster that supports TLS with a Certificate from a ClusterIssuer of letsencrypt.

However, we have been unable to get this working, We have crt-manager set up in the cluster but for some reason, I don’t think that the operator is using it.

I tried a few workarounds to make it happen:

  1. workaround 1 - I download the ca.pam from letsencrypt website and I thought to use it as the ca.crt in the *-ssl and/or *-ssl-internal secrets. - psmdb stay stuck with initializing state with errors: No SSL certificate provided by peer; connection rejected

  2. workaround 2 - try to create my own Certificate with the replica set nodes as DNS and ClusterIssuer of Letsencrypt using cert-manager, then I took the tls.crt and tls.key and combine them with the ca.pam from letsencrypt (from workaround 1)

certificate.yaml:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: cluster-tls
namespace: mongodb
spec:
secretName: minimal-cluster-tls
issuerRef:
kind: ClusterIssuer
name: “letsencrypt”
dnsNames:
- stg-mongo-psmdb-rs0-0.comapny.com
- stg-mongo-psmdb-rs0-1.comapny.com
- stg-mongo-psmdb-rs0-2.comapny.com

result: psmdb still stuck on initializing state with errors:

The server certificate does not match the remote host name" and Failed to authenticate

and

{“client”“,“mechanism”:“MONGODB-X509”,“user”:“CN=stg-mongo-psmdb-rs0-0.comapny.com”,“db”:”$external",“error”:{“code”:11,“codeName”:“UserNotFound”,“errmsg”:“Could not find user "CN=stg-mongo-psmdb-rs0-0.comapny.com" for db "$external"”}}}

I don’t quite sure if it’s related, but as far as I understand when the clusterServiceDNSMode is set to External , then I can reach it from outside with this command:
mongo "mongodb://databaseAdmin:databasepassword@DNS:27017,DNS:27017,DNS:27017/admin?replicaSet=rs0"

apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDB
metadata:
name: stg-mongo-psmdb
finalizers:
- delete-psmdb-pods-in-order
- delete-psmdb-pvc
spec:
clusterServiceDNSMode: “External”
unmanaged: false
allowUnsafeConfigurations: false
updateStrategy: Never
crVersion: 1.15.0
image: percona/percona-server-mongodb:5.0.15-13
imagePullPolicy: Always
upgradeOptions:
apply: disabled
schedule: “0 2 * * *”
secrets:
users: stg-eng-mongo-users
ssl: my-custom-ssl
sslInternal: my-custom-ssl-internal
replsets:

  • name: rs0
    size: 3
    expose:
    enabled: true
    exposeType: LoadBalancer
    serviceAnnotations:
    networking.gke.io/load-balancer-type: “Internal”
    volumeSpec:
    persistentVolumeClaim:
    resources:
    requests:
    storage: 3Gi
    sharding:
    enabled: false

any ideas on how I can do it with our own Certificate, I don’t want to use the operator Issuer because I cannot share the certificate(*.pem) with all the clients that will need to contact DB.

Thanks.

2

Just playing around with a similar case.

I guess you should define

ssl: my-custom-ssl

with the secretName of your certificate. Does that help?

Kind regards

3

Hey @rdxmbr,

indeed, I defined ssl: my-custom-ssl with the secretName of my certificate.
it was one of my workarounds.

did you manage to create a replicaset cluster with TLS and connect with the connection string tls=true ?

Thank you

4

it’s work in progress here …

5

… but I miss a full docs how to work with cert-manager and the cr.yaml . Transport encryption (TLS/SSL) - Percona Operator for MongoDB does only explain how to install cert-manager (I have it running, so that’s not the problem) , but not how to use those certificates.

In my case I am actually trying to change the ssl: parameter and not changing sslInternal. This should be possible.

But: I am not using letsencrypt for that, but GitHub - smallstep/step-issuer: ⚙️A certificate issuer for cert-manager using step certificates CA and my own step-ca running. With this, it is possible to expand the certificates with the kubernetes-internal dns names.

6

Regarding the cert-manager, as far as I understood, the certificate creates when the server is initialized ad the first time with the name of the cluster, it creates a secret, the issue with that secret is the ca.crt. it’s own issuer and not the clusterissuer for letsencrypt

When you change only the SSL you manage to log in to the cluster with tls=true as replicaSet?

7

Regarding the cert-manager, as far as I understood, the certificate creates when the server is initialized ad the first time with the name of the cluster, it creates a secret, the issue with that secret is the ca.crt . it’s own issuer and not the clusterissuer for letsencrypt

this is also what I understood.

When you change only the SSL you manage to log in to the cluster with tls=true as replicaSet?

No, there are child-fields of ssl: , changing the certificate definitions.

8

@Avner_Zini

seems you need to define

  usages:
    - signing
    - key encipherment
    - server auth
    - client auth
  subject:
    organizations:
      - PSMDB

in your certificate. (not tested yet). Will you give it a try?