Description:
The MongoDB Instance Summary dashboard in PMM 3.6.0 shows “No data” on several panels when a mongos service is selected. The Service Name dropdown includes mongos services, so users expect data to appear, but the panel queries only use mongod-specific metrics (mongodb_mongod_metrics_document_total, mongodb_mongod_metrics_query_executor_total) that are not available on mongos instances.
Additionally, when using --enable-all-collectors on mongos via the Percona Operator’s mongosParams, the indexstats collector queries $indexStats across all shards, causing the same collection+index combination to be returned multiple times (once per shard) with identical labels. This results in duplicate metric errors for mongodb_indexstats_accesses_ops.
Steps to Reproduce:
Issue 1: Instance Summary “No data” on mongos
-
Deploy a sharded MongoDB cluster using Percona Operator with PMM enabled
-
Navigate to MongoDB > Instance Summary dashboard
-
Select a mongos service in the
Service Namedropdown -
Observe “No data” on Document Operations and Query Efficiency panels
Issue 2: --enable-all-collectors duplicate metric error on mongos
-
Set
mongosParams: "--enable-all-collectors"in the PerconaServerMongoDB CR -
Wait for mongos pods to restart with the new PMM config
-
Check pmm-client container logs:
kubectl logs <mongos-pod> -c pmm-client -
Observe repeated duplicate metric errors
Version:
- PMM Server: 3.6.0
- PMM Client: 3.6.0 (percona/pmm-client:3.6.0)
- Percona Server for MongoDB: 8.0.8-3
- Percona Operator for MongoDB: 1.21.1
- Platform: AWS EKS
Logs:
pmm-client container log from mongos pod (Issue 2):
level=ERROR source=http_error_logger.go:47 msg="error gathering metrics:"
error="[from Gatherer #2] collected metric \"mongodb_indexstats_accesses_ops\"
{ label:{name:\"cl_role\" value:\"mongos\"}
label:{name:\"collection\" value:\"lookup_franchise_no_stage_2026_01_09\"}
label:{name:\"database\" value:\"cdp\"}
label:{name:\"key_name\" value:\"hashed_id_idx\"}
untyped:{value:0}}
was collected before with the same name and label values"
total_errors=36
This error repeats for every collection/index combination across all shards (36 errors per scrape cycle).
Expected Result:
Issue 1:
- Instance Summary should either filter mongos services out of the Service Name dropdown, or conditionally switch queries to mongos-compatible metrics (e.g.,
mongodb_mongod_op_latencies_*which works on both mongod and mongos), or hide panels that are not applicable for mongos.
Issue 2:
--enable-all-collectorson mongos should work without duplicate metric errors. The exporter should either deduplicate index stats from multiple shards, or add ashardlabel to differentiate them.
Actual Result:
Issue 1:
-
Document Operations panel shows “No data” because it queries
mongodb_mongod_metrics_document_totalwhich is mongod-only. -
Query Efficiency panel shows “No data” for the same reason.
-
Other panels like Operation Latencies work correctly because
mongodb_mongod_op_latencies_*is collected from mongos as well.
Issue 2:
-
36 duplicate metric errors per scrape cycle, one for each collection+index combination across shards.
-
The indexstats collector on mongos queries all shards via
$indexStats, returning the same collection+index from multiple shards with identical labels, causing the Prometheus collector to reject duplicates.
Additional Information:
-
Cluster topology: 3 shards x 3 replset nodes + 3 mongos + 3 config servers
-
The Router Summary dashboard correctly displays mongos metrics. This issue is specific to Instance Summary.
-
Current workaround for Issue 2: only use
--enable-all-collectorsonmongodParams, notmongosParams:
yaml
pmm:
enabled: true
mongodParams: "--enable-all-collectors --environment=prod"
mongosParams: "--environment=prod"
- Some
mongodb_mongod_*metrics are collected from mongos (e.g.,op_latencies), while others are not (e.g.,metrics_document_total). This naming inconsistency makes it difficult to determine which metrics are available for mongos without trial and error.