Not the answer you need?
Register and ask your own question!

MongoDB PMM Installation

bobhobobho ContributorInactive User Role Beginner
Hello, I am trying to install PMM tool to monitor a MongoDB client in the same network, the MongoDB client is on a ubuntu trusty(so I am using pmm-client_1.1.1-1.trusty_amd64.deb to install), and the PMM Server is on a VirtualBox(so I am using PMM-Server-2017-02-16.ova to monitor) !! But after I install them, I can NOT see any graph on the PMM Server ... here is the "pmm-admin check-network" output :
PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

* System Time
Server | 2017-03-14 11:30:28 +0000 GMT
Client | 2017-03-14 11:30:52 +0800 CST
Time Drift | 28776s

Time is out of sync. Please make sure the server time is correct to see the metrics.


* Connection: Client --> Server

SERVER SERVICE STATUS

Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 861.946µs
Request duration | 4.927953ms
Full round trip | 5.789899ms


* Connection: Client <-- Server





SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD





linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run 'pmm-admin list' to verify).
If it's running, check out the logs /var/log/pmm-*.log

When all endpoints are down but 'pmm-admin list' shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: http://172.16.18.102/prometheus/targets

It seems that the service status is DOWN ... what should I do ? Can you help me ?

Comments

  • MykolaMykola Percona Percona Staff Role
    Hi bobho,

    1. can you run and check network again?
    also please run command

    2. do you have any kind of firewall?
    it is needed to allow connection from pmm-server to pmm-client 42000, 42003 ports.
  • bobhobobho Contributor Inactive User Role Beginner
    Hi! Mykola :

    Here is the pmm-admin start -all and pmm-admin list output and the check network again :

    [email protected]:~$ sudo pmm-admin start --all
    OK, 2 services are started.
    [email protected]:~$ sudo pmm-admin list
    pmm-admin 1.1.1

    PMM Server | 172.16.18.102
    Client Name | pc-0725
    Client Address | 172.16.18.63
    Service Manager | linux-systemd






    SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS





    linux:metrics pc-0725 42000 YES -
    mongodb:metrics pc-0725 42003 YES localhost:27017

    [email protected]:~$ sudo pmm-admin check-network
    PMM Network Status

    Server Address | 172.16.18.102
    Client Address | 172.16.18.63

    * System Time
    Server | 2017-03-14 13:25:43 +0000 GMT
    Client | 2017-03-14 13:46:14 +0800 CST
    Time Drift | 27569s

    Time is out of sync. Please make sure the server time is correct to see the metrics.


    * Connection: Client --> Server

    SERVER SERVICE STATUS

    Consul API OK
    Prometheus API OK
    Query Analytics API OK

    Connection duration | 2.364165ms
    Request duration | 1.428277ms
    Full round trip | 3.792442ms


    * Connection: Client <-- Server





    SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD





    linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
    mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

    When an endpoint is down it may indicate that the corresponding service is stopped (run 'pmm-admin list' to verify).
    If it's running, check out the logs /var/log/pmm-*.log

    When all endpoints are down but 'pmm-admin list' shows they are up and no errors in the logs,
    check the firewall settings whether this system allows incoming connections from server to address:port in question.

    Also you can check the endpoint status by the URL: http://172.16.18.102/prometheus/targets

    [email protected]:~$

    2. I had closed the firewall on the VirtualBox PC where the PMM Server located !!

    And here I also attached the URL of http://172.16.18.102/prometheus/targets output for your reference :)
  • MykolaMykola Percona Percona Staff Role
    Do you have any firewall on MongoDB side?
  • bobhobobho Contributor Inactive User Role Beginner
    The MongoDB is a Linux running Ubuntu Trusty(14.04) on our LAN, and I am pretty sure there is NO firewall of any kind on it :)

    In the http://172.16.18.102/prometheus/targets ) and find a lot of data in them, so I guess the 2 ports are open !!

    But the Endpoint links of http://localehost:9100/metrics for linux and the Endpoint link of http://localhost:9090/prometheus/metrics for prometheus can NOT be reached :confused:
  • MykolaMykola Percona Percona Staff Role
    it is strange why shows DOWN, but according to targets page everything is working fine

    do you have any other errors anywhere?
    do you see MongoDB graphs in grafana?
  • bobhobobho Contributor Inactive User Role Beginner
    In the /var/log/pmm-linux-metrics-42000.log and /var/log/pmm-mongodb-metrics-42003.log, there are some errors about TLS handshake error ...

    [email protected]:~$ cat /var/log/pmm-linux-metrics-42000.log
    time="2017-03-14T10:44:41+08:00" level=info msg="Starting node_exporter (version=1.1.1, branch=master, revision=2d78e22000779d63c714011e4fb30c65623b9c77)" source="node_exporter.go:170"
    time="2017-03-14T10:44:41+08:00" level=info msg="Build context (go=go1.7.4, user=, date=)" source="node_exporter.go:171"
    time="2017-03-14T10:44:41+08:00" level=info msg="Enabled collectors:" source="node_exporter.go:190"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - filefd" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - loadavg" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - meminfo" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - stat" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - vmstat" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - diskstats" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - filesystem" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - netdev" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - netstat" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - time" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg=" - uname" source="node_exporter.go:192"
    time="2017-03-14T10:44:41+08:00" level=info msg="HTTPS/TLS is enabled" source="node_exporter.go:235"
    time="2017-03-14T10:44:41+08:00" level=info msg="Listening on 172.16.18.63:42000" source="node_exporter.go:238"
    2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:55114: tls: first record does not look like a TLS handshake
    2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:57094: tls: first record does not look like a TLS handshake
    2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:40756: tls: first record does not look like a TLS handshake
    2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:54292: tls: first record does not look like a TLS handshake
    2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:54728: tls: first record does not look like a TLS handshake
    2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:57892: tls: first record does not look like a TLS handshake
    2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:38342: tls: first record does not look like a TLS handshake

    [email protected]:~$ cat /var/log/pmm-mongodb-metrics-42003.log
    ### Warning: the exporter is in beta/experimental state and field names are very
    ### likely to change in the future and features may change or get removed!
    ### See: https://github.com/percona/mongodb_exporter for updates
    mongodb_exporter version: 1.1.1, git commit hash: c890996b9133a3288530da4a9fb6e6fad067b55f
    HTTPS/TLS is enabled
    Listening on 172.16.18.63:42003
    2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:59072: tls: first record does not look like a TLS handshake
    2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:32820: tls: first record does not look like a TLS handshake
    2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:44714: tls: first record does not look like a TLS handshake
    2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:58246: tls: first record does not look like a TLS handshake
    2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:58686: tls: first record does not look like a TLS handshake
    2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:33618: tls: first record does not look like a TLS handshake
    2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:42300: tls: first record does not look like a TLS handshake
    [email protected]:~$
  • MykolaMykola Percona Percona Staff Role
    can you share output of the following command?
    docker ps | grep pmm-server
    
  • bobhobobho Contributor Inactive User Role Beginner
  • MykolaMykola Percona Percona Staff Role
    Oh, sorry.
    ssh user is 'admin' (we are adding this to documentation in PMM-650)

    can you ssh to VirtualBox and synchronize time?
    and reboot...
  • bobhobobho Contributor Inactive User Role Beginner
    What is the password ?
  • MykolaMykola Percona Percona Staff Role
    immediately after the first virtualbox instance start, web interface asks username, password and public ssh key.
    did you configured ssh key?
  • bobhobobho Contributor Inactive User Role Beginner
    " on the screen) !!

    Can you show me how to configure SSH Key ?

    First input "admin" for the Username, then ...
  • MykolaMykola Percona Percona Staff Role
    Do you have ssh key?

    we have two options:
    1. remove current instance and create new
    2. set ssh key via REST API (command line)
  • bobhobobho Contributor Inactive User Role Beginner
  • MykolaMykola Percona Percona Staff Role
    public ssh key should looks like one long line which starts with "ssh-rsa"
    curl -X POST -d '{"Key": "ssh-rsa log-log-line some-comment"}' http://login:password&#64;PMM-SERVER-IP/configurator/v1/sshkey
    
    as you can see, you need the current http credentials

    this command should return ssh key fingerprint as response
This discussion has been closed.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.