ProxySQL added to "PPM Inventory" but the status remain N/A

1

I’ve added a ProxySQL Service to the PMM Inventory by running

pmm-admin add proxysql --username=radmin --password=radmin --host=172.30.18.212 --port=6032 --service-name=proxysql-service --debug

on the ProxySQL instance

The command debug output is as follows:

ubuntu@ip-172-30-18-212:~$ pmm-admin add proxysql --username=radmin --password=radmin --host=172.30.18.212 --port=6032 --service-name=proxysql-service --debug
DEBUG 2025-04-15 07:25:10.008537764Z: POST /local/Status HTTP/1.1
Host: 127.0.0.1:7777
User-Agent: Go-http-client/1.1
Content-Length: 3
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{}

DEBUG 2025-04-15 07:25:10.009743076Z: HTTP/1.1 200 OK
Content-Length: 1138
Content-Type: application/json
Date: Tue, 15 Apr 2025 07:25:10 GMT
Grpc-Metadata-Content-Type: application/grpc

{
“agent_id”: “/agent_id/04b9c01f-e53a-4786-aaf4-5bbb2607a5d9”,
“runs_on_node_id”: “/node_id/5283e9c5-8545-4245-a942-6ba7899ad6d4”,
“server_info”: {
“url”: “https://api_key:eyJrIjoiZFc2Qkt5QzZFMzUyMk93VGtxbDVTVHJmcHcyN25wVUsiLCJuIjoicG1tLWFnZW50LWlwLTE3Mi0zMC0xOC0yMTItODM4ODkwMTkzNTk0NDU2Njk3OCIsImlkIjoxfQ==@172.30.18.50:443/”,
“insecure_tls”: true,
“connected”: true,
“version”: “2.44.0”,
“latency”: null,
“clock_drift”: null
},
“agents_info”: [
{
“agent_id”: “/agent_id/47f5aaea-eb7e-42ac-a145-efdded352a42”,
“agent_type”: “NODE_EXPORTER”,
“status”: “RUNNING”,
“listen_port”: 42000,
“process_exec_path”: “/usr/local/percona/pmm2/exporters/node_exporter”
},
{
“agent_id”: “/agent_id/c08d974a-65ae-4b5d-ad5d-7d3718968815”,
“agent_type”: “VM_AGENT”,
“status”: “RUNNING”,
“listen_port”: 42001,
“process_exec_path”: “/usr/local/percona/pmm2/exporters/vmagent”
}
],
“config_filepath”: “/usr/local/percona/pmm2/config/pmm-agent.yaml”,
“agent_version”: “2.44.0”,
“node_name”: “ip-172-30-18-212”,
“connection_uptime”: 100
}
DEBUG 2025-04-15 07:25:10.010142783Z: POST /local/Status HTTP/1.1
Host: 127.0.0.1:7777
User-Agent: Go-http-client/1.1
Content-Length: 3
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{}

DEBUG 2025-04-15 07:25:10.011135461Z: HTTP/1.1 200 OK
Content-Length: 1138
Content-Type: application/json
Date: Tue, 15 Apr 2025 07:25:10 GMT
Grpc-Metadata-Content-Type: application/grpc

{
“agent_id”: “/agent_id/04b9c01f-e53a-4786-aaf4-5bbb2607a5d9”,
“runs_on_node_id”: “/node_id/5283e9c5-8545-4245-a942-6ba7899ad6d4”,
“server_info”: {
“url”: “https://api_key:eyJrIjoiZFc2Qkt5QzZFMzUyMk93VGtxbDVTVHJmcHcyN25wVUsiLCJuIjoicG1tLWFnZW50LWlwLTE3Mi0zMC0xOC0yMTItODM4ODkwMTkzNTk0NDU2Njk3OCIsImlkIjoxfQ==@172.30.18.50:443/”,
“insecure_tls”: true,
“connected”: true,
“version”: “2.44.0”,
“latency”: null,
“clock_drift”: null
},
“agents_info”: [
{
“agent_id”: “/agent_id/47f5aaea-eb7e-42ac-a145-efdded352a42”,
“agent_type”: “NODE_EXPORTER”,
“status”: “RUNNING”,
“listen_port”: 42000,
“process_exec_path”: “/usr/local/percona/pmm2/exporters/node_exporter”
},
{
“agent_id”: “/agent_id/c08d974a-65ae-4b5d-ad5d-7d3718968815”,
“agent_type”: “VM_AGENT”,
“status”: “RUNNING”,
“listen_port”: 42001,
“process_exec_path”: “/usr/local/percona/pmm2/exporters/vmagent”
}
],
“config_filepath”: “/usr/local/percona/pmm2/config/pmm-agent.yaml”,
“agent_version”: “2.44.0”,
“node_name”: “ip-172-30-18-212”,
“connection_uptime”: 100
}
DEBUG 2025-04-15 07:25:10.011413976Z: POST /v1/management/ProxySQL/Add HTTP/1.1
Host: 172.30.18.50:443
User-Agent: Go-http-client/1.1
Content-Length: 303
Accept: application/json
Authorization: Basic YXBpX2tleTpleUpySWpvaVpGYzJRa3Q1UXpaRk16VXlNazkzVkd0eGJEVlRWSEptY0hjeU4yNXdWVXNpTENKdUlqb2ljRzF0TFdGblpXNTBMV2x3TFRFM01pMHpNQzB4T0MweU1USXRPRE00T0Rrd01Ua3pOVGswTkRVMk5qazNPQ0lzSW1sa0lqb3hmUT09
Content-Type: application/json
Accept-Encoding: gzip

{“node_id”:“/node_id/5283e9c5-8545-4245-a942-6ba7899ad6d4”,“service_name”:“proxysql-service”,“address”:“172.30.18.212”,“port”:6032,“pmm_agent_id”:“/agent_id/04b9c01f-e53a-4786-aaf4-5bbb2607a5d9”,“username”:“radmin”,“password”:“radmin”,“metrics_mode”:“AUTO”,“disable_collectors”:null,“log_level”:“warn”}

DEBUG 2025-04-15 07:25:10.042744393Z: HTTP/1.1 200 OK
Content-Length: 604
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json
Date: Tue, 15 Apr 2025 07:25:10 GMT
Grpc-Metadata-Content-Type: application/grpc
Pragma: no-cache
Server: nginx
Strict-Transport-Security: max-age=63072000; includeSubdomains;
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block

{
“service”: {
“service_id”: “/service_id/5b0a2717-cf67-42f0-ba48-5e580937e5f8”,
“service_name”: “proxysql-service”,
“node_id”: “/node_id/5283e9c5-8545-4245-a942-6ba7899ad6d4”,
“address”: “172.30.18.212”,
“port”: 6032
},
“proxysql_exporter”: {
“agent_id”: “/agent_id/33e91160-ca16-46f9-9da4-0963a138a547”,
“pmm_agent_id”: “/agent_id/04b9c01f-e53a-4786-aaf4-5bbb2607a5d9”,
“service_id”: “/service_id/5b0a2717-cf67-42f0-ba48-5e580937e5f8”,
“username”: “radmin”,
“push_metrics_enabled”: true,
“status”: “UNKNOWN”,
“log_level”: “warn”
}
}
DEBUG 2025-04-15 07:25:10.042982397Z: Result: &management.addProxySQLResult{Service:(*proxy_sql.AddProxySQLOKBodyService)(0xc0000a2e60)}
DEBUG 2025-04-15 07:25:10.042999707Z: Error:
ProxySQL Service added.
Service ID : /service_id/5b0a2717-cf67-42f0-ba48-5e580937e5f8
Service name: proxysql-service

The ProxySQL version is

ubuntu@ip-172-30-18-212:~$ proxysql --version
ProxySQL version 2.7.2-96-g5816a7d, codename Truls

The status of the added ProxySQL service in the PMM Inventory remains N/A and I can’t get any metrics for it

Will highly appreciate your help on fixing this

2

Hello @Harutyun_Grigoryan

Please check pmm-agent log for further investigation,

journalctl -u pmm-agent.service -f

3

Thanks @lalit.choudhary

All I see there is

ubuntu@ip-172-30-18-212:~$ journalctl -u pmm-agent.service -f
Apr 16 03:00:00 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:00:00.325+00:00” level=error msg=“Failed to connect to 172.30.18.50:443: timeout.” component=client
Apr 16 03:00:17 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:00:17.052+00:00” level=error msg=“Failed to connect to 172.30.18.50:443: timeout.” component=client
Apr 16 03:00:39 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:00:39.900+00:00” level=error msg=“Failed to connect to 172.30.18.50:443: timeout.” component=client
Apr 16 03:01:03 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:03.422+00:00” level=error msg=“Failed to connect to 172.30.18.50:443: timeout.” component=client
Apr 16 03:01:19 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:19.619+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor
Apr 16 03:01:19 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:19.620+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor
Apr 16 03:01:19 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:19.620+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor
Apr 16 03:01:20 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:20.887+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor
Apr 16 03:01:20 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:20.887+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor
Apr 16 03:01:20 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:20.888+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor

The connection errors at 3 AM are caused by Percona EC2 and ProxySQL being paused in the night and started at that time

4

Hello @Harutyun_Grigoryan

Apr 16 03:01:03 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:03.422+00:00” level=error msg=“Failed to connect to 172.30.18.50:443: timeout.” component=client
Apr 16 03:01:19 ip-172-30-18-212 pmm-agent[739]: time=“2025-04-16T03:01:19.619+00:00” level=error msg=“Failed to reserve port: no free port.” component=supervisor

Seems like there is a connectivity issue to pmm-server, Please check connectivity to the pmm server using telnet for 443 port,

telnet 172.30.18.50 443

5

@lalit.choudhary

ubuntu@ip-172-30-18-212:~$ telnet 172.30.18.50 443
Trying 172.30.18.50...
Connected to 172.30.18.50.
Escape character is '^]'.
^]
telnet> 

quit
HTTP/1.1 400 Bad Request
Server: nginx
Date: Thu, 17 Apr 2025 13:48:10 GMT
Content-Type: text/html
Content-Length: 150
Connection: close
Strict-Transport-Security: max-age=63072000; includeSubdomains;

<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx</center>
</body>
</html>
Connection closed by foreign host.
ubuntu@ip-172-30-18-212:~$

The logs reflect that the ProxySQL and Percona EC2-s are restarted daily at 3 AM UTC
I get similar logs every day