Not the answer you need?
Register and ask your own question!

pmm-admin config not sticking

joehuerschjoehuersch ContributorCurrent User Role Beginner
For some reason the pmm-admin setting seem to not being set in the config files, and as such, no subsequent commands are working.

Both Deb and apt-get install result in this same case on a Debian 8.3 install with percona server 5.6 running locally, passworded root.

Comments

  • weberweber Advisor Inactive User Role Beginner
    Very strange, can you check if /usr/local/percona/pmm-client/pmm.yml file is being created?
  • joehuerschjoehuersch Contributor Current User Role Beginner
    It would appear that it is not on the clients end.
    # ls /usr/local/percona/pmm-client/
    mongodb_exporter  mysqld_exporter  node_exporter  proxysql_exporter
    
  • weberweber Advisor Inactive User Role Beginner
    So no file created. I wonder whether it is something specific on Debian 8.3...

    Do you have other working client? You may try to copy pmm.yml from there and change client address and client name to see if it works.
  • joehuerschjoehuersch Contributor Current User Role Beginner
    I do have a working client on a CentOS 6.8 system- although it's not working as well as it should, the query/innoDB data seems to not be coming through from MySQL5.6 despite the mysql:query monitoring being active and having full root level access to the instance.

    I will copy the pmm.yml file over to the debian install and reply with the results.
  • joehuerschjoehuersch Contributor Current User Role Beginner
    I seems to be working, but despite changing the values for the bind and client addresses, as well as the client name, the uptime, number of CPUs, and memory usage appear to be the same as the CentOS6.8 server after starting the monitors on the debian8.3 system.
    # pmm-admin add mysql --user root --password xxxxxxxxxxxxxxxx --disable-tablestats-limit 10000
    [linux:metrics] OK, now monitoring this system.
    [mysql:metrics] OK, now monitoring MySQL metrics using DSN root:***@unix(/var/run/mysqld/mysqld.sock)
    [mysql:queries] OK, now monitoring MySQL queries from slowlog using DSN root:***@unix(/var/run/mysqld/mysqld.sock)
    
  • joehuerschjoehuersch Contributor Current User Role Beginner
    The debian8.3 system seems to be having issues with the mysql:metrics monitor.
    time="2016-12-15T14:06:21-05:00" level=info msg="Starting mysqld_exporter (version=1.0.7, branch=master, revision=4a4c53fb313fb1883bcbd464f53c83f73b336100)" source="mysqld_exporter.go:679"
    time="2016-12-15T14:06:21-05:00" level=info msg="Build context (go=go1.7.4, user=, date=)" source="mysqld_exporter.go:680"
    time="2016-12-15T14:06:21-05:00" level=info msg="HTTP basic authentication is enabled" source="mysqld_exporter.go:709"
    time="2016-12-15T14:06:21-05:00" level=info msg="HTTPS/TLS is enabled" source="mysqld_exporter.go:724"
    time="2016-12-15T14:06:21-05:00" level=info msg="Listening on 192.168.111.108:42002" source="mysqld_exporter.go:727"
    2016/12/15 14:06:58 http: TLS handshake error from 192.168.111.108:45891: tls: first record does not look like a TLS handshake
    2016/12/15 14:06:58 http: TLS handshake error from 192.168.111.108:45892: tls: first record does not look like a TLS handshake
    2016/12/15 14:07:20 http: TLS handshake error from 192.168.111.108:45914: tls: first record does not look like a TLS handshake
    2016/12/15 14:07:20 http: TLS handshake error from 192.168.111.108:45915: tls: first record does not look like a TLS handshake
    
  • weberweber Advisor Inactive User Role Beginner
    Those TLS errors can be ignored.
  • joehuerschjoehuersch Contributor Current User Role Beginner
    Now that I've removed the monitors and rebuilt them, I see no data points in grafana for the centOS 6.8 system for mysql query stats despite setting innodb logging settings and a slow query log. QAN has data, but prometheus and graphana seem to not be getting anything.
    # pmm-admin list
    pmm-admin 1.0.7
    
    PMM Server      | 98.76.54.32:8080 (password-protected)
    Client Name     | centos68
    Client Address  | 12.34.56.78
    Service Manager | unix-systemv
    
    -------------- -------------------------------- ----------- -------- ------------------------------------------ ------------------------
    SERVICE TYPE   NAME                             LOCAL PORT  RUNNING  DATA SOURCE                                OPTIONS
    -------------- -------------------------------- ----------- -------- ------------------------------------------ ------------------------
    mysql:queries  centos68  -           YES      admin:***@unix(/var/lib/mysql/mysql.sock)  query_source=perfschema
    linux:metrics  centos68  42000       YES      -                     
    mysql:metrics  centos68  42002       YES      admin:***@unix(/var/lib/mysql/mysql.sock)
    
    # pmm-admin check-network
    PMM Network Status
    
    Server Address | 98.76.54.32:8080
    Client Address | 12.34.56.78
    
    * System Time
    Server     | 2016-12-19 08:40:59 -0500 EST
    Client     | 2016-12-19 08:40:59 -0500 EST
    Time Drift | OK
    
    
    * Connection: Client --> Server
    -------------------- -------
    SERVER SERVICE       STATUS
    -------------------- -------
    Consul API           OK
    Prometheus API       OK
    Query Analytics API  OK
    
    Connection duration | 67.923482ms
    Request duration    | 72.511934ms
    Full round trip     | 140.435416ms
    
    
    * Connection: Client <-- Server
    -------------- -------------------------------- --------------------- ------- ---------- ---------
    SERVICE TYPE   NAME                             REMOTE ENDPOINT       STATUS  HTTPS/TLS  PASSWORD
    -------------- -------------------------------- --------------------- ------- ---------- ---------
    linux:metrics  centos68  12.34.56.78:42000  OK      YES        YES
    mysql:metrics  centos68  12.34.56.78:42002  OK      YES        YES
    
  • weberweber Advisor Inactive User Role Beginner
    What do you see in Grafana and /prometheus/targets page?
  • joehuerschjoehuersch Contributor Current User Role Beginner
    Graphana is only showing query data for the Debian8.3 system which has percona server 5.6, prometheus is showing all targets are up. the CentOS6.8 system has MySQL 5.6 on it, with slow query log configured.
  • weberweber Advisor Inactive User Role Beginner
    Then I see no problem querying metrics at /prometheus/graph web interface. You may try to query for any metric.
    If you can query Prometheus, there is no reason why it is not shown in Grafana.
    Do you have all dashboards empty?
    The last thing to check - please verify that the server time (underlying host where docker runs) and your local machine time (where you run web browser) is OK (what timezone is set - does not matter, the time should be simply correct on that timezone).
  • joehuerschjoehuersch Contributor Current User Role Beginner
    All the dashboards have something in them for their related function, but the the CentOS6.8 server does not have any mysql stats in prometheus, only the debian 8.3 system has mysql stats in there- Could this be due to the fact that it's percona server instead of mysql 5.6 community?
  • MykolaMykola Percona Percona Staff Role
    joehuersch wrote: »
    Could this be due to the fact that it's percona server instead of mysql 5.6 community?
    No.

    Are PMM server and PMM client 1.0.7 version?
  • joehuerschjoehuersch Contributor Current User Role Beginner
    server is using percona/pmm-latest, built just a few weeks ago so I'd assume 1.0.7; and clients are all pmm-client 1.0.7
  • MykolaMykola Percona Percona Staff Role
    is selinux disabled?
This discussion has been closed.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.