Pmm-agent "404 Not Found" trying to register with server

Percona Monitoring and Management (PMM) PMM 3.x
1

Hi,
Just venturing in to the Percona monitoring world. I’ve got the server set up in an Ubuntu 24.04 VM with the script that installs docker and the latest container (pmm-server:3). That’s all working fine as far as I can tell. I’m able to get to the web UI. I’m trying to set up a pmm-agent (pmm-agent 2.24, mysql 5.7, Ubuntu 16.04) to connect to the server and am consistently getting a “404 Not Found” when running the base configuration.

  • The agent is running.
  • The config file is owned by pmm-agent.
  • I’ve checked basic connectivity between client-server and get a 200OK when running a curl from client to server on tcp/443;
  • Getting a mysql version string when runing a netcat test from server to client.

I’ve just been following the instructions here along with other

# pmm-agent setup --server-insecure-tls  --server-address=se.rv.er.ip:443 --server-username=admin --server
-password=SUPER_GOOD_PASSWD --debug
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/node_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/mysqld_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/mongodb_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/postgres_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/proxysql_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/rds_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/azure_exporter  component=setup
INFO[2025-06-06T14:34:01.376-04:00] Using /usr/local/percona/pmm2/exporters/vmagent  component=setup
Checking local pmm-agent status...
DEBU[2025-06-06T14:34:01.377-04:00] POST /local/Status HTTP/1.1
Host: 127.0.0.1:7777
User-Agent: Go-http-client/1.1
Content-Length: 3
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{}
  component=setup
DEBU[2025-06-06T14:34:01.379-04:00] HTTP/1.1 200 OK
Content-Length: 191
Content-Type: application/json
Date: Fri, 06 Jun 2025 18:34:01 GMT
Grpc-Metadata-Content-Type: application/grpc

{
  "agent_id": "",
  "runs_on_node_id": "",
  "server_info": null,
  "agents_info": [
  ],
  "config_filepath": "/usr/local/percona/pmm2/config/pmm-agent.yaml",
  "agent_version": "2.24.0"
}  component=setup
DEBU[2025-06-06T14:34:01.379-04:00] Status error: <nil>                           component=setup
pmm-agent is running.
Registering pmm-agent on PMM Server...
DEBU[2025-06-06T14:34:01.380-04:00] POST /v1/management/Node/Register HTTP/1.1
Host: se.rv.er.ip:443
User-Agent: Go-http-client/1.1
Content-Length: 235
Accept: application/json
Authorization: Basic YWRtaW46OUNtdmk0Vzl5QlJhZU1tRENNZ3NaQjhF
Content-Type: application/json
Accept-Encoding: gzip

{"node_type":"GENERIC_NODE","node_name":"mysql-server-host.name","address":"cl.ie.nt.ip","machine_id":"/machine_id/92c78b7bbd5d4ad7b2345859b89c3dd3","distro":"linux","metrics_mode":"AUTO","disable_collectors":null}
  component=setup
DEBU[2025-06-06T14:34:01.435-04:00] HTTP/1.1 404 Not Found
Content-Length: 116
Connection: keep-alive
Content-Type: application/json
Date: Fri, 06 Jun 2025 18:33:56 GMT
Server: nginx
Strict-Transport-Security: max-age=63072000; includeSubdomains;

{
  "code": 5,
  "message": "Endpoint not found: /v1/management/Node/Register, http method: POST",
  "details": []
}  component=setup
DEBU[2025-06-06T14:34:01.435-04:00] Register error: &node.RegisterNodeDefault{_statusCode:404, Payload:(*node.RegisterNodeDefaultBody)(0xc0004785c0)}  component=setup
Failed to register pmm-agent on PMM Server: .
2

Hi, seems like you are trying to connect PMM 2 Client to PMM 3 Server. Can you please check PMM Server version?

3

Yep. I had the exact same thought over the weekend. I was operating on the assumption was that 3 was backwards compatible… but then my now relaxed-brain determined that it might not be. :slight_smile:

4

PMM 3 Server has limited backward compatibility with PMM 2 Client. Basically it supports compatibility for metrics delivery from PMM 2 clients to PMM 3 server. It was done for those who upgrade their servers and it take time to upgrade all PMM 2 Clients.

5

But I’m guessing that registration itself does not work.

I installed a pmm2 server, registration is working fine. I’m unfortunately limited by the OSes that are installed on this MySQL cluster, they only seem to support the pmm2 client.

6

Yes, registration and all pmm-admin commands doesn’t work