Compiled PMM2 client on ARM, Installed but can't find mysqld_exporter

Percona Monitoring and Management (PMM) PMM 2.x
1

This is what pmm2-agent shows (pmm-agent --config-file=/data/pmm2/pmm-agent.yaml &)

/data/pmm2 is softlinked to /usr/local/percona/pmm2

It also sends some data to PMM server (MySQL service, slow queries), but not the client-host itself. No data in MySQL section / OS / InnoDB etc.,

Any pointers would be highly appreciated.

Thank you for your time!

INFO[2023-04-26T17:07:56.193+00:00] Process: waiting 26.719954408s.               agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:07:56.194+00:00] Sending status: WAITING (port 42000).         agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:08:00.132+00:00] Sending 4989 buckets.                         agentID=/agent_id/7175c76d-496c-42dd-8d1a-80d3a312592c component=agent-builtin type=qan_mysql_perfschema_agent
INFO[2023-04-26T17:08:10.285+00:00] Sending status: STARTING (port 42002).        agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
WARN[2023-04-26T17:08:10.285+00:00] Process: failed to start: fork/exec /usr/local/percona/pmm2/exporters/mysqld_exporter: no such file or directory.  agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:08:10.286+00:00] Process: waiting 28.654842666s.               agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:08:10.286+00:00] Sending status: WAITING (port 42002).         agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:08:22.917+00:00] Sending status: STARTING (port 42000).        agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
WARN[2023-04-26T17:08:22.917+00:00] Process: failed to start: fork/exec /usr/local/percona/pmm2/exporters/vmagent: no such file or directory.  agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:08:22.917+00:00] Process: waiting 29.023687109s.               agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:08:22.917+00:00] Sending status: WAITING (port 42000).         agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:08:38.949+00:00] Sending status: STARTING (port 42002).        agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
WARN[2023-04-26T17:08:38.949+00:00] Process: failed to start: fork/exec /usr/local/percona/pmm2/exporters/mysqld_exporter: no such file or directory.  agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:08:38.949+00:00] Process: waiting 31.876425424s.               agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:08:38.949+00:00] Sending status: WAITING (port 42002).         agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:08:51.944+00:00] Sending status: STARTING (port 42000).        agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
WARN[2023-04-26T17:08:51.944+00:00] Process: failed to start: fork/exec /usr/local/percona/pmm2/exporters/vmagent: no such file or directory.  agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:08:51.944+00:00] Process: waiting 30.752203807s.               agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:08:51.944+00:00] Sending status: WAITING (port 42000).         agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
INFO[2023-04-26T17:09:00.141+00:00] Sending 4989 buckets.                         agentID=/agent_id/7175c76d-496c-42dd-8d1a-80d3a312592c component=agent-builtin type=qan_mysql_perfschema_agent
INFO[2023-04-26T17:09:10.840+00:00] Sending status: STARTING (port 42002).        agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
WARN[2023-04-26T17:09:10.840+00:00] Process: failed to start: fork/exec /usr/local/percona/pmm2/exporters/mysqld_exporter: no such file or directory.  agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:09:10.841+00:00] Process: waiting 31.854132396s.               agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:09:10.841+00:00] Sending status: WAITING (port 42002).         agentID=/agent_id/2eed3bac-870e-46ed-ae65-39a7a4ee10da component=agent-process type=mysqld_exporter
INFO[2023-04-26T17:09:22.697+00:00] Sending status: STARTING (port 42000).        agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
WARN[2023-04-26T17:09:22.698+00:00] Process: failed to start: fork/exec /usr/local/percona/pmm2/exporters/vmagent: no such file or directory.  agentID=/agent_id/7cb63f96-3035-464f-9aab-b76e5843c068 component=agent-process type=vm_agent
2

Just to add, followed this guide to compile: Steps to compile PMM v2 client on AWS EC2 ARM (github.com)

node_exporter compiled 18M (the only binary by node_exporter) and no other support file was there.

PMM Server is installed via docker on 64bit CentOS as per single line command installation.

3

Anyone, can help …? Or atleast share which component owns these exporters, so I can focus on that direction.

Thanks once again.

4

It appears the gist you followed is incomplete. I did not see any steps to compile the mysqld_exporter. You will need to compile the mysql exporter for ARM. Each *_exporter is its own binary that must be compiled for ARM. You should be able to expand the steps in the gist to include the mysqld_exporter.

5

Can you share a command for the same?

6

Okay, my bad. Need to clone all repos and build them, got it! Thanks for your help!

8

As new PMM released, trying to build RPMs from source. But I am getting this error, any hints?

➜  /data rpmbuild --define 'dist .el9' --rebuild pmm2-client-2.37.0-6.src.rpm
Installing pmm2-client-2.37.0-6.src.rpm
warning: line 5: Possible unexpanded macro in: Version:        %{version}
warning: line 6: Possible unexpanded macro in: Release:        %{release}.el9
error: /home/ec2-user/rpmbuild/SPECS/pmm2-client.spec: line 11: Too many levels of recursion in macro expansion. It is likely caused by recursive macro declaration.

Just for reference, this is the OS

➜  /data uname -a
Linux ip-xxx.ec2.internal 5.14.0-299.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Thu Apr 13 10:49:45 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

Any hints would be much appreciated!