PMM 2.37.1 Qan mysql slowlog agent is always waiting

Percona Monitoring and Management (PMM) PMM 2.x
1

Hello,

I’m using PMM Server on SLES 15.4, one of my Database is running on Ubuntu 22.04 (MYSQL 8.x.x).
When I try to add the Database the slowlog agent is not working correctly, the status is always waiting.

The version of my client is 2.37.1

I’ve enabled using SET GLOBAL slow_query_log = ‘ON’; and set long_query_time to 0, in order to log everything.

Using this command I tried to check if it’s working: sudo tail /var/lib/mysql/marvintestsystem-slow.log

Output:
Time: 2023-06-26T12:04:39.395746Z
User@Host: pmm[pmm] @ localhost [127.0.0.1] Id: 1816
Query_time: 0.000217 Lock_time: 0.000001 Rows_sent: 0 Rows_examined: 0
SET timestamp=1687781079;
/*!50700 SELECT conn_status.channel_name as channel_name, conn_status.service_state as IO_thread, applier_status.service_state as SQL_thread, 1 as info FROM performance_schema.replication_connection_status AS conn_status JOIN performance_schema.replication_applier_status_by_worker AS applier_status ON applier_status.channel_name = conn_status.channel_name WHERE conn_status.service_state = ‘ON’ and (SELECT SUBSTRING(@@VERSION,1,1) = 5) */;
Time: 2023-06-26T12:04:40.773238Z
User@Host: root[root] @ localhost Id: 1819
Query_time: 0.000010 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 1
SET timestamp=1687781080;
administrator command: Quit;

When I try to re add the service it’s still showing waiting.

When I execute sudo docker exec 78eec6651f61 pmm-admin status to check the agent status this is displayed:

Agents:
/agent_id/1af8d489-a566-4bb8-a6bf-3da5d99cb8e8 vmagent Running 42000
/agent_id/1fe675c2-6fe0-4d15-9e5e-f2de98e966c4 node_exporter Running 42001
/agent_id/292b0e31-b1d2-4881-b4d0-647794bdd9e5 postgresql_pgstatmonitor_agent Waiting 0
/agent_id/5d65f5ab-7265-42ae-9eee-cdda77ab6db9 mysqld_exporter Running 42011
/agent_id/acb27f9a-a9ef-45d6-b126-b61755e1300f mysql_slowlog_agent Waiting 0
/agent_id/bfb9d329-3f1e-49de-b22b-9574611f7db6 postgres_exporter Running 42009

Any ideas?

2

Hi @Marvin_Stelter
So, it looks like you running pmm-client in docker?
If this is the case - make sure the slow log is available on pmm-client docker, and also check the permissions for pmm-agent user on this file - can the user read it?

3

Hi @Roma_Novikov ,
I installed pmm-server using docker. Then I added the service from pmm-agent, the error message could not be connected to the pmm-server server.
After that, inside the container, i add service. The obtained results are:
image

Can you help me to resolve it? Thank you!

4

@trongdat.nguyen Could you please verify whether slow log was enabled? Can you produce the following commands?

  1. MariaDB [(none)]> show global variables like ‘%slow_query_log%’;
  2. MariaDB [(none)]> show global variables like ‘%log_out%’;
  3. MariaDB [(none)]> show global variables like ‘%long_query_time%’;
5

Here is my config about slow log:
image

6

Check if slow_log is enabled, check if Docker has access to the slow log file, check if the slow log file is accessible for the current user. That resolved my error

7

Check if the slow_query_log_file has chmod sest to 755 (That worked for me)

then start your client like this

sudo docker run -d
–name pmm-client
–network= “host” `
-e PMM_AGENT_SERVER_ADDRESS=
-e PMM_AGENT_SERVER_USERNAME=admin
-e PMM_AGENT_SERVER_PASSWORD=
-e PMM_AGENT_SERVER_INSECURE_TLS=1
-e PMM_AGENT_SETUP=1
-e PMM_AGENT_CONFIG_FILE=config/pmm-agent.yaml
-v /var/lib/mysql:/var/lib/mysql
percona/pmm-client:2

and now add your database, everything should work.