Hello,
we test the functionality pmm.
We have only 1 server : -
PMM Server (CentOS Linux release 7.7.1908 (Core) : [URL]http://89.200.136.175:8081/[/URL]
PMM Client - 89.200.136.175
For the Slow Query check for QAN PMM on the PMM Client :
slow_query_log_file=/var/log/mysql-slow.log
slow_query_log=ON
long_query_time=0.01
log_slow_admin_statements=ON
log_slow_slave_statements=ON
ON The Client:
[root@inkpraa5 ~]# pmm-admin list
pmm-admin 1.1.3
PMM Server | 89.200.136.175:8081
Client Name | inkpraa5.memset.netClient Address | 89.200.136.175
Service Manager | linux-systemd
SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS
mysql:queries inkpraa5.memset.net - YES aalabels_monitor:@unix(/var/lib/mysql/mysql.sock) query_source=perfschema, query_examples=true
linux:metrics inkpraa5.memset.net 42000 YES -
mysql:metrics inkpraa5.memset.net 42002 YES aalabels_monitor:@unix(/var/lib/mysql/mysql.sock)
[root@inkpraa5 ~]# pmm-admin check-network
PMM Network Status
Server Address | 89.200.136.175:8081
Client Address | 89.200.136.175
-
System Time
NTP Server (0.pool.ntp.org) | 2020-03-13 10:24:42 +0000 GMT
PMM Server | 2020-03-13 10:24:42 +0000 GMT
PMM Client | 2020-03-13 10:24:42 +0000 GMT
PMM Server Time Drift | OK
PMM Client Time Drift | OK
PMM Client to PMM Server Time Drift | OK -
Connection: Client → Server
SERVER SERVICE STATUS
Consul API OK
Prometheus API OK
Query Analytics API OK
Connection duration | 254.426µs
Request duration | 664.149µs
Full round trip | 918.575µs
- Connection: Client ← Server
SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD
linux:metrics inkpraa5.memset.net 89.200.136.175:42000 DOWN YES -
mysql:metrics inkpraa5.memset.net 89.200.136.175:42002 DOWN YES -
When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log
When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.
Also you can check the endpoint status by the URL: [url]http://89.200.136.175:8081/prometheus/targets[/url]
[root@inkpraa5 log]# cat pmm-mysql-metrics-42002.log
time=“2020-03-12T09:31:06Z” level=info msg=“Starting mysqld_exporter (version=1.1.3, branch=master, revision=4a4c53fb313fb1883bcbd464f53c83f73b336100)” source=“mysqld_exporter.go:679”
time=“2020-03-12T09:31:06Z” level=info msg=“Build context (go=go1.8, user=, date=)” source=“mysqld_exporter.go:680”
time=“2020-03-12T09:31:06Z” level=info msg=“HTTPS/TLS is enabled” source=“mysqld_exporter.go:724”
time=“2020-03-12T09:31:06Z” level=info msg=“Listening on 89.200.136.175:42002” source=“mysqld_exporter.go:727”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.info_schema.innodb_metrics: Error 1227: Access denied; you need (at least one of) the PROCESS privilege(s) for this operation” source=“mysqld_exporter.go:435”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.slave_status: Error 1227: Access denied; you need (at least one of) the SUPER, REPLICATION CLIENT privilege(s) for this operation” source=“mysqld_exporter.go:477”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.perf_schema.eventswaits: Error 1142: SELECT command denied to user ‘aalabels_monitor’@‘localhost’ for table ‘events_waits_summary_global_by_event_name’” source=“mysqld_exporter.go:489”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.perf_schema.file_events: Error 1142: SELECT command denied to user ‘aalabels_monitor’@‘localhost’ for table ‘file_summary_by_event_name’” source=“mysqld_exporter.go:495”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.perf_schema.tablelocks: Error 1142: SELECT command denied to user ‘aalabels_monitor’@‘localhost’ for table ‘table_lock_waits_summary_by_table’” source=“mysqld_exporter.go:501”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.perf_schema.tableiowaits: Error 1142: SELECT command denied to user ‘aalabels_monitor’@‘localhost’ for table ‘table_io_waits_summary_by_table’” source=“mysqld_exporter.go:579”
time=“2020-03-12T09:31:06Z” level=error msg=“Error scraping for collect.perf_schema.indexiowaits: Error 1142: SELECT command denied to user ‘aalabels_monitor’@‘localhost’ for table ‘table_io_waits_summary_by_index_usage’” source=“mysqld_exporter.go:585”
time=“2020-03-12T12:27:33Z” level=info msg=“Starting mysqld_exporter (version=1.1.3, branch=master, revision=4a4c53fb313fb1883bcbd464f53c83f73b336100)” source=“mysqld_exporter.go:679”
2020/03/13 09:52:21 http: TLS handshake error from 89.200.136.175:55970: tls: first record does not look like a TLS handshake
2020/03/13 10:24:42 http: TLS handshake error from 89.200.136.175:53802: tls: first record does not look like a TLS handshake
[root@inkpraa5 log]# cat pmm-mysql-queries-0.log
Version: percona-qan-agent 1.1.3
Basedir: /usr/local/percona/qan-agent
PID: 151754
API: 89.200.136.175:8081/qan-api
UUID: 196c4c948415433540723086884db2c4
2020/03/13 08:44:22.932996 main.go:165: Starting agent…
2020/03/13 08:44:22.935802 main.go:346: Agent is ready
2020/03/13 08:44:22.939176 main.go:207: API is ready
2020/03/13 08:44:51.620525 WARNING agent ws error: EOF
2020/03/13 08:44:51.620659 WARNING agent Lost connection to API
2020/03/13 08:44:51.620796 WARNING log Lost connection to API
2020/03/13 08:44:54.621487 WARNING agent-ws websocket.Dial ws://89.200.136.175:8081/qan-api/agents/196c4c948415433540723086884db2c4/cmd: dial tcp 89.200.136.175:8081: getsockopt: connection refused
2020/03/13 08:44:54.621596 WARNING log-ws websocket.Dial ws://89.200.136.175:8081/qan-api/agents/196c4c948415433540723086884db2c4/log: dial tcp 89.200.136.175:8081: getsockopt: connection refused
2020/03/13 08:45:00.001133 ERROR qan-analyzer-5b93639d Error 1142: SELECT command denied to user ‘aalabels_monitor’@'localhost’for table ‘events_statements_summary_by_digest’
2020/03/13 08:45:01.623976 WARNING agent-ws websocket.Dial ws://89.200.136.175:8081/qan-api/agents/196c4c948415433540723086884db2c4/cmd: dial tcp 89.200.136.175:8081: getsockopt: connection refused
2020/03/13 08:45:01.624055 WARNING log-ws websocket.Dial ws://89.200.136.175:8081/qan-api/agents/196c4c948415433540723086884db2c4/log: dial tcp 89.200.136.175:8081: getsockopt: connection refused
2020/03/13 08:46:00.002680 ERROR qan-analyzer-5b93639d Error 1142: SELECT command denied to user ‘aalabels_monitor’@'localhost’for table ‘events_statements_summary_by_digest’
‘events_statements_summary_by_digest’
[root@inkpraa5 ~]# docker exec -ti pmm-server bash
[root@c0ae057562b1 opt]# curl [url]http://89.200.136.175:42002/metrics[/url]
curl: (7) Failed connect to 89.200.136.175:42002; Connection timed out
[root@c0ae057562b1 opt]# date
Fri Mar 13 10:48:08 UTC 2020
Do you have the idea for this problem ?
Thank you for your help,
Regards,
Umar
