Node_exporter can not collect system related metrics

Percona Monitoring and Management (PMM) PMM 2.x
1

I am not sure what the specific PMM version when this issue emerges after I upgrade the PMM server(as I have done several upgrades)

symptom:
Currently my PMM server is 2.37.1 and the pmm-agent has the following versions:

  • 2.12.0
  • 2.25.0
  • 2.26.0
  • 2.27.0
  • 2.28.0
  • 2.32.0
  • 2.33.0
  • 2.34.0
  • 2.35.0

And I found pmm-agent below 2.28.0 works well, the system related metrics like cup,disk i/o,network can be collected and displayed on the dashboards. But versions larger than 2.28.0(inclusive) do not work. The related dashboards are empty.

Could someone advise on this? Are there any incompatible changes between PMM servers and pmm-agent?

BTW: by restarting the pmm-agent binary can fix this, but the issue will raise again some time in the future.

It should have something related to the tmp directory /tmp/node_exporter as I found when the node_exporter can not collect metrics, this temporary directory does not exist. But when I restart pmm-agent, this directory appears. And the node_exporter command references this path: e.g.

node_exporter ... --web.config=/tmp/node_exporter/agent_id/xxxxxxxxx-f1ca-41ee-ac7b-xxxxxxxx/webConfigPlaceholder ...
2

Hello @Luke03011,

Regarding the versions, there shouldn’t be any noticeable limitations. However, it seems that Log in - Percona JIRA may have impacted you. Our team is actively working on resolving this issue, and we hope to have a fix available in the next release.