jamoser
November 26, 2021, 11:25pm
1
Can anyone tell my why this error appears on the mongos ?
{“t”:{“$date”:“2021-11-26T23:13:44.032+00:00”},“s”:“W”, “c”:“NETWORK”, “id”:23238, “ctx”:“ShardRegistry”,“msg”:“The server certificate does not match the remote host name”,“attr”:{“remoteHost”:“xxx-mongodb01-cluster-rs01-0.xxx-mongodb01-cluster-rs01.dev-mongodb01.svc.cluster.local”,“certificateNames”:"SAN(s): localhost, my-cluster-name-rs0, my-cluster-name-rs0.psmdb, my-cluster-name-rs0.psmdb.svc.cluster.local, *.my-cluster-name-rs0, *.my-cluster-name-rs0.psmdb, *.my-cluster-name-rs0.psmdb.svc.cluster.local, my-cluster-name-mongos, my-cluster-name-mongos.psmdb, my-cluster-name-mongos.psmdb.svc.cluster.local, *.my-cluster-name-mongos, *.my-cluster-name-mongos.psmdb, *.my-cluster-name-mongos.psmdb.svc.cluster.local, my-cluster-name-cfg, my-cluster-name-cfg.psmdb, my-cluster-name-cfg.psmdb.svc.cluster.local, *.my-cluster-name-cfg, *.my-cluster-name-cfg.psmdb, *.my-cluster-name-cfg.psmdb.svc.cluster.local, "}}
In my opinion
xxx-mongodb01-cluster-rs01-0.xxx-mongodb01-cluster-rs01.dev-mongodb01.svc.cluster.local
should match with
*.my-cluster-name-rs0.psmdb.svc.cluster.local
?
I’ve deleted the cluster incl SSL certs and restarted - nothing changed. This config worked but suddenly turned bad. I have the same config also for different kubernetes namespaces and they all work fine.
Hey @jamoser ,
how did you get into this situation?
In my opinion
xxx-mongodb01-cluster-rs01-0.xxx-mongodb01-cluster-rs01.dev-mongodb01.svc.cluster.local
should match with
*.my-cluster-name-rs0.psmdb.svc.cluster.local
?
They should not match.
dev-mongodb01.svc.cluster.local vs psmdb.svc.cluster.local - two different domains
xxx-mongodb01-cluster-rs01 vs my-cluster-name-rs0 - same here
jamoser
November 29, 2021, 2:28pm
3
The only thing that has changed is, that the rs01-1/2/3 pods were moved to another node pool (eg. different nodes).
But the same happened to other replicasets ?!
To fix this issue, do I have to delete the cfg disks ? and start the cluster again ?
Curretnly I just dont see where the configuration regarding *.my-cluster-name-rs0.psmdb.svc.cluster.local is stored.
@jamoser I’m still not clear on how you got there.
What did you have? (cr, k8s config)
jamoser
November 30, 2021, 12:28am
5
The setup is as follows and there were no changes (to my knowledge) :
get pods :
xxx-mongodb01-cluster-cfg-0 1/1 Running 0 3d1h
xxx-mongodb01-cluster-cfg-1 1/1 Running 0 3d1h
xxx-mongodb01-cluster-cfg-2 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-c2vbq 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-c9m54 0/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-wmvbd 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-0 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-1 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-2 1/1 Running 0 3d1h
percona-server-mongodb-operator-d859b69b6-m52dr 1/1 Running 0 3d1h
after a few secs
xxx-mongodb01-cluster-cfg-0 1/1 Running 0 3d1h
xxx-mongodb01-cluster-cfg-1 1/1 Running 0 3d1h
xxx-mongodb01-cluster-cfg-2 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-c2vbq 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-c9m54 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-wmvbd 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-0 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-1 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-2 1/1 Running 0 3d1h
percona-server-mongodb-operator-d859b69b6-m52dr 1/1 Running 0 3d1h
after a few secs
xxx-mongodb01-cluster-cfg-0 1/1 Running 0 3d1h
xxx-mongodb01-cluster-cfg-1 1/1 Running 0 3d1h
xxx-mongodb01-cluster-cfg-2 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-c2vbq 0/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-c9m54 1/1 Running 0 3d1h
xxx-mongodb01-cluster-mongos-5764c4c87-wmvbd 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-0 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-1 1/1 Running 0 3d1h
xxx-mongodb01-cluster-rs01-2 1/1 Running 0 3d1h
percona-server-mongodb-operator-d859b69b6-m52dr 1/1 Running 0 3d1h
As you can see the mongos stop to be available for a few seconds and then they are again due to this cert matching issue.
The funny thing is the whole cluster is “available” even though some mongos are not available !
Actually my original question was: where is the config regarding the cert matching stored ? How can I fix this ?
If you want to renew the certs manually - the easiest way would be to delete and recreate or alter the Secrets where certs are stored. Operator would do the rest.
By default they are in the following secrets:
my-cluster-name-ssl
You can read about cert generation here: Transport encryption (TLS/SSL) - Percona Operator for MongoDB
jamoser
January 13, 2022, 10:48pm
9
Out of the box the cert does not fit but do not affect the functionality. The error always appears unless the cert gets adjusted to the host name.
Response of @Sergey_Pronin solved the issue.
→ Please close this ticket.