Getting warning message on Pmm2.7

Percona Monitoring and Management (PMM) PMM 2.x
1

Hi Team,
We are monitoring mongoDb 4.2 by pmm2 but few warning messages are coming in log 

couldn’t retrieve data from cursor connection(127.0.0.1:27017[-99]) incomplete read of message header: read tcp 127.0.0.1:36222->127.0.0.1:27017: i/o timeout #033[33m***D#033[0m=/agent_id/13fc7034-704c-4b66-be5e-e165f50480e2 #033[33mcomponent#033[0m=agent-builtin #033[33mdb#033[0m=bob-qa4 #033[33mtype#033[0m=qan_mongodb_profiler_agent

Please have a look
Thanks

2

I saw a similar message once when I was playing with Mongo as a container instead of a bare install (I had disabled the firewall to allow me to register the exporter and then later rebooted and the rule wasn’t made permanent so started blocking connection).  Looks like this is the local pmm-admin agent trying to connect for QAN data and getting denied but I’d expect to see similar entries for the mongo_exporter trying to scrape as well.  Are you able to upgrade to 2.9.1?  Getting client and server at the latest version would at least rule out something that’s already been fixed in the last few releases.  

1 Like
3

Hi Steve,
i upgraded pmm to 2.91 but still we are having same error .

Agent ID: /agent_id/6892b59a-e570-4d51-9ef6-d1bf9833df17Node ID : /node_id/21d567c4c-1d1d-4733-a1ab-70d4118b454a
PMM Server:        URL    : https://pmm-server-ip:443/        Version: 2.9.1
PMM Client:        Connected        : true        Time drift       : 1.476634ms        Latency          : 1.073369ms        pmm-admin version: 2.9.1        pmm-agent version: 2.9.1Agents:        /agent_id/6e732314-fbe8-432e-b6d3-2e5ae5c2c31d node_exporter Running        /agent_id/70a55aa4-e80b-4bbc-a4f7-eb676b5d6d3 mongodb_profiler_agent Running        /agent_id/d330b911-5cb4-4adf-8d53-1260ee501f14 mongodb_exporter Running

1 Like
4

Hi @Gajendra
Even I saw this error but after re-adding the service, it’s gone. later I don’t see the same issue.
Can you try same?

Try the following,

  • remove pmm2-client and directory ‘/usr/local/percona/pmm2’

  • install pmm2-client version again (same as pmm server version)

  • and add instance in monitoring again.

1 Like
5

Seen a similar issue with 2.16.0 Here is the bug report: [PMM-6912] mongodb_exporter_qan hung due server overload - Percona JIRA

1 Like