I was trying to setup the monitoring of a MongoDB sharded cluster and, although I have been able to configure the replica sets (shards), I am struggling to properly work with the MongoDB routers.
pmm-client connect using a user (previously used with MMS) called mms with clusterMonitor and readAnyDatabase roles, but as soon as I activate the mongodb:metrics service, the mongos log file is filled with messages like the following (coming 1 block per second):
2016-12-13T15:51:48.622+0000 I ACCESS [conn559] Successfully authenticated as principal mms on admin
2016-12-13T15:51:48.623+0000 I NETWORK [conn559] query on admin.$cmd: { query: “1”, help: 1 } failed to: mongocfg-hxvm-001:27019 (10.7.49.239) exception: “query” had the wrong type. Expected Object, found String
2016-12-13T15:51:48.623+0000 I NETWORK [conn559] query on admin.$cmd: { query: “1”, help: 1 } failed to: mongocfg-pgvm-001:27019 (10.42.24.102) exception: “query” had the wrong type. Expected Object, found String
2016-12-13T15:51:48.623+0000 I NETWORK [conn559] query on admin.$cmd: { query: “1”, help: 1 } failed to: mongocfg-oyvm-001:27019 (10.32.24.102) exception: “query” had the wrong type. Expected Object, found String
2016-12-13T15:51:48.623+0000 W NETWORK [conn559] db exception when initializing on config, current connection state is { state: { conn: “SyncClusterConnection [mongocfg-hxvm-001:27019 (10.7.49.239),mongocfg-pgvm-001:27019 (10.42.24.102),mongocfg-oyvm-001:2…”, vinfo: “config:mongocfg-hxvm-001:27019,mongocfg-pgvm-001:27019,mongocfg-oyvm-001:27019”, cursor: “(empty)”, count: 0, done: false }, retryNext: false, init: false, finish: false, errored: false } :: caused by :: 6 all servers down/unreachable when querying: mongocfg-hxvm-001:27019,mongocfg-pgvm-001:27019,mongocfg-oyvm-001:27019
Checking the 42003 daemon log, it displays the following entries:
E1213 15:51:53.106705 21846 sharding_topology.go:85] Failed to execute find query on ‘config.chunks’!
E1213 15:51:53.532863 21846 sharding_topology.go:112] Failed to execute find query on ‘config.collections’!
E1213 15:51:53.624214 21846 sharding_topology.go:85] Failed to execute find query on ‘config.chunks’!
However the mms user is able to issue queries against the config database:
mongos> db.auth(‘mms’,‘XXXXXXXX’)
1
mongos> use config
switched to db config
mongos> db.chunks.findOne()
{
“id" : "eva_hsapiens_grch37.variants_1_2-chr"5"start_506294”,
“lastmod” : Timestamp(15246, 0),
“lastmodEpoch” : ObjectId(“574806fe54d3eec4a2fb2ec9”),
“ns” : “eva_hsapiens_grch37.variants_1_2”,
“min” : {
“chr” : “5”,
“start” : 506294
},
“max” : {
“chr” : “5”,
“start” : 616474
},
“shard” : “ebiclu01rs04”
}
Can anyone please help?
Regards,
Alessio