MongoDB cluster not getting populated on PMM2 dashboard

Percona Monitoring and Management (PMM) PMM 2.x
1

Hi all,

We are facing a strange issue where we are not able to see one of the mongodb cluster on PMM2 dashboard. It’s a three node replicaset and I can see that all services/agents/nodes are successfully added on PMM Inventory List.

To have this cluster on PMM MongoDB, is there any other configuration needed on PMM dashboard? Or could there be some issue with mongodb-exporter? We are able to see all the three nodes on “OS/ Nodes overview” graph but not in MongoDB.

Any assistance is really appreciated!

Thank you!

Harsh

1 Like
2

Hi, try the metrics explorer to see if there are any metrics with the name mongodb. If not you have a problem either with the PMM user privileges or you didn’t run the pmm-admin add mongodb command.
Hope that helps!

1 Like
3

Hi Ivan, Thank you for a kind response. I do see mongodb metrics under explorer for other clusters. It’s just this one particular cluster that is having issues where mongodb metrics not visible on pmm.

I checked the curl command for mongodb_exporter agent-ID locally on primary node, and I do see it is able to fetch mongodb metrics. Somehow these metrics are not getting displayed on pmm

$ pmm-admin list
Service type        Service name                   Address and port        Service ID
MongoDB             <service_name>        127.0.0.1:27017         /service_id/c1b7f891-fc78-4894-94eb-6d6ace7ecf61

Agent type                    Status           Metrics Mode        Agent ID                                              Service ID                                              Port
pmm_agent                     Connected                            /agent_id/15e3b0bf-29d9-4b8e-9dbe-b9fb85d4c985                                                                0
node_exporter                 Running          push                /agent_id/1868f8ea-cb1a-4f83-8e8d-c35f4808ceca                                                                11001
mongodb_exporter              Running          push                /agent_id/bcf098b8-73c1-4837-928a-4dac964ff3e3        /service_id/c1b7f891-fc78-4894-94eb-6d6ace7ecf61        11003
mongodb_profiler_agent        Running                              /agent_id/73d9db6b-0187-41bd-ae61-6f3e80e0914b        /service_id/c1b7f891-fc78-4894-94eb-6d6ace7ecf61        0
vmagent                       Running          push                /agent_id/3374d223-9764-4882-9fcb-ed5b28aa208d

$ time curl http://127.0.0.1:11003/metrics -u "pmm:/agent_id/bcf098b8-73c1-4837-928a-4dac964ff3e3"
. . . .
< getting all mongodb metrics here >
. . . .
real	0m7.107s
user	0m0.011s
sys	0m0.138s

Thanks!

Harsh

1 Like
4

Hello, if the exporter is showing the metrics locally, that would point to some connectivity issue to PMM server. Can you remove the client and install again? also make sure you use the same client version as PMM server.

1 Like
5

We tried reinstalling the client still no luck with mongodb metrics. All our clusters are running below pmm server client version and they are running fine without any issue. It’s just this one cluster that has issues. I might be wrong but since OS/node metrics is working it shouldn’t be connectivity issue. I believe there should be some issue with mongodb-exporter

PMM Server: 2.31.0
pmm-admin version: 2.32.0
pmm-agent version: 2.32.0

Checking pmm-agent.log we intermittently see below error messages. Not sure if this can cause pmm to stop sending metrics to server?

INFO[2022-12-13T16:14:56.743+00:00] 2022-12-13T16:14:56.742Z error VictoriaMetrics/lib/promscrape/scrapework.go:355 error when scraping "http://127.0.0.1:11003/metrics" from job "mongodb_exporter_agent_id_bcf098b8-73c1-4837-928a-4dac964ff3e3_hr" . . . . . . . error when scraping "http://127.0.0.1:11003/metrics" with timeout 4.5s: timeout agentID=/agent_id/3374d223-9764-4882-9fcb-ed5b28aa208d component=agent-process type=vm_agent

1 Like
6

Since we have large number of collections. After disabling collstats --disable-collectors=collstats we are no longer facing timeout errors. Dashboard is loading data as expected.

Thanks!