For some reason the pmm-admin setting seem to not being set in the config files, and as such, no subsequent commands are working.
Both Deb and apt-get install result in this same case on a Debian 8.3 install with percona server 5.6 running locally, passworded root.
Very strange, can you check if /usr/local/percona/pmm-client/pmm.yml file is being created?
It would appear that it is not on the clients end.
# ls /usr/local/percona/pmm-client/
mongodb_exporter mysqld_exporter node_exporter proxysql_exporter
So no file created. I wonder whether it is something specific on Debian 8.3…
Do you have other working client? You may try to copy pmm.yml from there and change client address and client name to see if it works.
I do have a working client on a CentOS 6.8 system- although it’s not working as well as it should, the query/innoDB data seems to not be coming through from MySQL5.6 despite the mysql:query monitoring being active and having full root level access to the instance.
I will copy the pmm.yml file over to the debian install and reply with the results.
I seems to be working, but despite changing the values for the bind and client addresses, as well as the client name, the uptime, number of CPUs, and memory usage appear to be the same as the CentOS6.8 server after starting the monitors on the debian8.3 system.
# pmm-admin add mysql --user root --password xxxxxxxxxxxxxxxx --disable-tablestats-limit 10000
[linux:metrics] OK, now monitoring this system.
[mysql:metrics] OK, now monitoring MySQL metrics using DSN root:***@unix(/var/run/mysqld/mysqld.sock)
[mysql:queries] OK, now monitoring MySQL queries from slowlog using DSN root:***@unix(/var/run/mysqld/mysqld.sock)
The debian8.3 system seems to be having issues with the mysql:metrics monitor.
time="2016-12-15T14:06:21-05:00" level=info msg="Starting mysqld_exporter (version=1.0.7, branch=master, revision=4a4c53fb313fb1883bcbd464f53c83f73b336100)" source="mysqld_exporter.go:679"
time="2016-12-15T14:06:21-05:00" level=info msg="Build context (go=go1.7.4, user=, date=)" source="mysqld_exporter.go:680"
time="2016-12-15T14:06:21-05:00" level=info msg="HTTP basic authentication is enabled" source="mysqld_exporter.go:709"
time="2016-12-15T14:06:21-05:00" level=info msg="HTTPS/TLS is enabled" source="mysqld_exporter.go:724"
time="2016-12-15T14:06:21-05:00" level=info msg="Listening on 192.168.111.108:42002" source="mysqld_exporter.go:727"
2016/12/15 14:06:58 http: TLS handshake error from 192.168.111.108:45891: tls: first record does not look like a TLS handshake
2016/12/15 14:06:58 http: TLS handshake error from 192.168.111.108:45892: tls: first record does not look like a TLS handshake
2016/12/15 14:07:20 http: TLS handshake error from 192.168.111.108:45914: tls: first record does not look like a TLS handshake
2016/12/15 14:07:20 http: TLS handshake error from 192.168.111.108:45915: tls: first record does not look like a TLS handshake
Those TLS errors can be ignored.
Now that I’ve removed the monitors and rebuilt them, I see no data points in grafana for the centOS 6.8 system for mysql query stats despite setting innodb logging settings and a slow query log. QAN has data, but prometheus and graphana seem to not be getting anything.
# pmm-admin list
pmm-admin 1.0.7
PMM Server | 98.76.54.32:8080 (password-protected)
Client Name | centos68
Client Address | 12.34.56.78
Service Manager | unix-systemv
-------------- -------------------------------- ----------- -------- ------------------------------------------ ------------------------
SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS
-------------- -------------------------------- ----------- -------- ------------------------------------------ ------------------------
mysql:queries centos68 - YES admin:***@unix(/var/lib/mysql/mysql.sock) query_source=perfschema
linux:metrics centos68 42000 YES -
mysql:metrics centos68 42002 YES admin:***@unix(/var/lib/mysql/mysql.sock)
# pmm-admin check-network
PMM Network Status
Server Address | 98.76.54.32:8080
Client Address | 12.34.56.78
* System Time
Server | 2016-12-19 08:40:59 -0500 EST
Client | 2016-12-19 08:40:59 -0500 EST
Time Drift | OK
* Connection: Client --> Server
-------------------- -------
SERVER SERVICE STATUS
-------------------- -------
Consul API OK
Prometheus API OK
Query Analytics API OK
Connection duration | 67.923482ms
Request duration | 72.511934ms
Full round trip | 140.435416ms
* Connection: Client <-- Server
-------------- -------------------------------- --------------------- ------- ---------- ---------
SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD
-------------- -------------------------------- --------------------- ------- ---------- ---------
linux:metrics centos68 12.34.56.78:42000 OK YES YES
mysql:metrics centos68 12.34.56.78:42002 OK YES YES
What do you see in Grafana and /prometheus/targets page?
Graphana is only showing query data for the Debian8.3 system which has percona server 5.6, prometheus is showing all targets are up. the CentOS6.8 system has MySQL 5.6 on it, with slow query log configured.
Then I see no problem querying metrics at /prometheus/graph web interface. You may try to query for any metric.
If you can query Prometheus, there is no reason why it is not shown in Grafana.
Do you have all dashboards empty?
The last thing to check - please verify that the server time (underlying host where docker runs) and your local machine time (where you run web browser) is OK (what timezone is set - does not matter, the time should be simply correct on that timezone).
All the dashboards have something in them for their related function, but the the CentOS6.8 server does not have any mysql stats in prometheus, only the debian 8.3 system has mysql stats in there- Could this be due to the fact that it’s percona server instead of mysql 5.6 community?
No.
Are PMM server and PMM client 1.0.7 version?
server is using percona/pmm-latest, built just a few weeks ago so I’d assume 1.0.7; and clients are all pmm-client 1.0.7
is selinux disabled?
MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright © 2006 - 2021 Percona LLC. All rights reserved.