Not the answer you need?
Register and ask your own question!

PMM2 mysql exporter change port after each delete/add

Hello,

When I add the mysql export on my client, the port changes each time : 42001 then 42002 then 42003 ...

Prometheus manages to make the connection to get the metrics, only with port 42002, on the other ports I get the error "context deadline expected".

On my firewall, ports 42000 to 50000 are open.


Example of command :

pmm-admin add mysql [email protected] --query-source=slowlog --username=example --password=example

==> if I run a ps aux | grep pmm, the service runs on port 42001

pmm-admin remove mysql serverName-mysql

pmm-admin add mysql [email protected] --query-source=slowlog --username=example --password=example

==> if I run a ps aux | grep pmm, the service runs on port 42002


Someone manages to reproduce it? I don't know if it's a bug, or a configuration problem on my side.

Have a nice day

Tagged:

Comments

  • adivinhoadivinho Percona Percona Staff Role

    Hi Laurent,


    It's possible to monitor few services by the same pmm-agent. Each new exporter will be start on an available 420xx port.

    Could you check if any errors are shown in pmm agent log?

    /var/log/pmm-agent.log


  • mlaurentmlaurent Current User Role Novice

    Hi,

    I have this error in the logs for November 2nd :

    Nov 2 18:00:17 dba pmm-agent[2922]: #033[36mINFO#033[0m[2020-11-02T18:00:17.051+01:00] time="2020-11-02T18:00:17+01:00" level=info msg="Successfully scraped status with query: SHOW ALL SLAVES STATUS" source="slave_status.go:113" #033[36magentID#033[0m=/agent_id/be83475c-e4b3-4e80-8396-cdc77ecc224c #033[36mcomponent#033[0m=agent-process #033[36mtype#033[0m=mysqld_exporter
    Nov 2 18:00:17 dba pmm-agent[2922]: #033[36mINFO#033[0m[2020-11-02T18:00:17.051+01:00] time="2020-11-02T18:00:17+01:00" level=error msg="Error scraping for collect.custom_query.hr: mysql_perf_schema_replication_group_worker_5:error running query on database: mysql_perf_schema_replication_group_worker_5, Error 1146: Table 'performance_schema.replication_connection_status' doesn't exist:mysql_perf_schema_replication_group:error running query on database: mysql_perf_schema_replication_group, Error 1193: Unknown system variable 'server_uuid':mysql_perf_schema_replication_group_5:error running query on database: mysql_perf_schema_replication_group_5, Error 1193: Unknown system variable 'server_uuid':mysql_perf_schema:error running query on database: mysql_perf_schema, Error 1193: Unknown system variable 'server_uuid':mysql_perf_schema_5:error running query on database: mysql_perf_schema_5, Error 1193: Unknown system variable 'server_uuid':mysql_perf_schema_replication_group_worker:error running query on database: mysql_perf_schema_replication_group_worker, Error 1305: FUNCTION GTID_SUBTRACT does not exist" source="exporter.go:116" #033[36magentID#033[0m=/agent_id/be83475c-e4b3-4e80-8396-cdc77ecc224c #033[36mcomponent#033[0m=agent-process #033[36mtype#033[0m=mysqld_exporter
    

    The agent rebooted, it was running on port 42002 and I was receiving logs. It rebooted at 6pm last night on port 42001, and I'm not receiving anything anymore.

    Here are the current logs, no apparent error:

    Nov 3 16:19:00 dba pmm-agent[2922]: #033[36mINFO#033[0m[2020-11-03T16:19:00.001+01:00] Sending 66 buckets.              #033[36magentID#033[0m=/agent_id/58660e56-3e9a-4522-92e6-b07b2395b828 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_slowlog_agent
    Nov 3 16:20:00 dba pmm-agent[2922]: #033[36mINFO#033[0m[2020-11-03T16:20:00.001+01:00] Sending 92 buckets.              #033[36magentID#033[0m=/agent_id/58660e56-3e9a-4522-92e6-b07b2395b828 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_slowlog_agent
    Nov 3 16:21:00 dba pmm-agent[2922]: #033[36mINFO#033[0m[2020-11-03T16:21:00.000+01:00] Sending 30 buckets.              #033[36magentID#033[0m=/agent_id/58660e56-3e9a-4522-92e6-b07b2395b828 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_slowlog_agent
    

    I have my firewall open on the following ports : 42000:42020/tcp      ALLOW    Anywhere


    Thanks

  • adivinhoadivinho Percona Percona Staff Role

    Hi Laurent,

    Error for November 2nd has fixed in the next release

    https://jira.percona.com/browse/PMM-6650

    And it has no relation to the issue.

    Could you check if PMM server is still scraping the old port (42002) ?

    It's expected that server is notified by pmm-agent refer port changes.

    e.g.


Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.