'clusterAuthMode: x509' failing with "Could not find user "CN=mongod1" for db $external"

Greetings,
Using the image percona/percona-server-mongodb:4.2, I am attempting to enable clusterAuthMode: x509 on a replicaset. I have followed the instructions here, making sure my certificates conform to the requirements. Each member is failing to connect to the other two members. Looking through the logs, I’m seeing entries like this:

authenticate db: $external { authenticate: 1, mechanism: "MONGODB-X509", user: "CN=mongod1", $db: "$external" }
2020-09-11T19:15:50.644+0000 I ACCESS [conn10] Failed to authenticate CN=mongod1@$external from client 172.19.62.122:37102 with mechanism MONGODB-X509: UserNotFound: Could not find user "CN=mongod1" for db "$external"

This would suggest that a user with the same name as each connecting member’s CN is expected; I’ve scoured the mongodb documentation and though there is documentation talking about the need for clients to authenticate with an existing user, I see no indication that such a user is needed for internal/membership authentication, which db they should be created on with what privileges, etc.

Is such a user necessary for each member when using x509 auth mode (if so, please point me to the documentation I’ve apparently overlooked!), or is there perhaps something else I’m missing here?

Any help appreciated!

Here is my conf file: (edit: anyone know of a better way to display my conf file? when i wrapped it in code, it put it all in one line…)
begin
net:  port: 17017  bindIpAll: true  tls:    mode: requireTLS    CAFile: /certs/myCA.pem    certificateKeyFile: /certs/myCert.pemreplication:  oplogSizeMB: 4096  replSetName: set1  enableMajorityReadConcern: falsesetParameter:  enableLocalhostAuthBypass: trueprocessManagement:  fork: "false"storage:  dbPath: /data/db  engine: wiredTiger  journal:    enabled: true  wiredTiger:    collectionConfig:      blockCompressor: snappy    engineConfig:      directoryForIndexes: true      cacheSizeGB: 20security:  enableEncryption: true  encryptionKeyFile: /etc/mongodb.keyfile  authorization: enabled  clusterAuthMode: x509
end

To put it in a nutshell: does anyone know if a user is needed for each member of a replica set when using clusterAuthMode: x509? Seems to be the case, but I’m not finding anything in the documentation that states this, what the user permissions should be, etc.

Hello @spiffly423

Does ‘mongod1’ part of your “CN=mongod1” matches the hostname of the server?

Here is quote from the https://docs.mongodb.com/manual/tutorial/configure-x509-member-authentication/:

Either the Common Name (CN) or one of the Subject Alternative Name (SAN) entries must match the hostname of the server, used by the other members of the cluster.

@Igor_Solodovnikov thanks for your response. Yes, the CN had matched the hostname of the server.

Finally figured this out. We had hosts that were named in an unorthodox way, e.g. “mongod1.dc1” that the x509 authentication didn’t like. Once I dropped the .dc1 on the hostname, it played nice. We were probably breaking some convention by adding .dc1, etc. to the hostname.

Perhaps the reason why it was failing is because it assumed everything after the first . was part of the domain? e.g. it was stripping mongod1.dc1.my.domain down to mongod1 and then that of course failed to match with the actual hostname of mongod1.dc1.

At any rate, very happy to have it working now!