I’m seeing a weird issue. Only the first node I added to PMM monitoring is showing up in query analytics. The other nodes show their mysql and linux stats, but not their QAN/Slow log. in the pmm summary, the node that is showing up in QAN is named “pl-db14”… none of the other nodes show up.
Was previously running PMM 1.x and things were working fine, only after moving to PMM 2.x (2.3.0 to start then upgraded to 2.5.0) and I started experiencing this issue.
I opened a jire ticket for this: [PMM-5807] only one node showing up in query analytics - Percona JIRA
In there, you can download pmm-admin summary and also debug output from the PMM server itself. Wondering if anyone else has run into this.
I’ve tried basic stuff like removing/adding the mysql service (pmm-admin add mysql --query-source=slowlog --username=pmm --password=xxxx --disable-tablestats) … verified stuff is going to my slowlog… nothing is working so far.
any ideas?
I would look at it his way
- Is it collected correctly ?
- Is it sent ?
- Is it stored in the Database ?
- Is it Displayed
First I would check if your slow query log is getting queries as you expect. If you have not enable logging all queries you may not get all queries there… When I’d check
Apr 22 20:02:00 ts140i pmm-agent[3399]: INFO[2020-04-22T20:02:00.000-04:00] Sending 1 buckets. agentID=/agent_id/685a6400-4ef0-4b15-9644-94cc026c6aa3 component=agent-builtin type=qan_mysql_slowlog_agent
Do you have any Buckets being sent ? (look at journalctl -xe output)
If this works see if you get any data if you query clickhouse directly:
https://www.percona.com/blog/2020/03/30/advanced-query-analysis-in-percona-monitoring-and-management-with-direct-clickhouse-access/
Hi Michael,
I have the same issue with only one node showing QAN, did you manage to find out and resolve the problem by chance?
Ta - Stephen