MongoDB PMM Installation

Percona Monitoring and Management (PMM) PMM 1.x
#1

Hello, I am trying to install PMM tool to monitor a MongoDB client in the same network, the MongoDB client is on a ubuntu trusty(so I am using pmm-client_1.1.1-1.trusty_amd64.deb to install), and the PMM Server is on a VirtualBox(so I am using PMM-Server-2017-02-16.ova to monitor) !! But after I install them, I can NOT see any graph on the PMM Server … here is the “pmm-admin check-network” output :

PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

  • System Time
    Server | 2017-03-14 11:30:28 +0000 GMT
    Client | 2017-03-14 11:30:52 +0800 CST
    Time Drift | 28776s

Time is out of sync. Please make sure the server time is correct to see the metrics.

  • Connection: Client --> Server

SERVER SERVICE STATUS

Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 861.946µs
Request duration | 4.927953ms
Full round trip | 5.789899ms

  • Connection: Client <-- Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD

linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: http://172.16.18.102/prometheus/targets

It seems that the service status is DOWN … what should I do ? Can you help me ?

#2

Hi bobho,

  1. can you run and check network again?
    also please run command

  2. do you have any kind of firewall?
    it is needed to allow connection from pmm-server to pmm-client 42000, 42003 ports.

#3

Hi! Mykola :

Here is the pmm-admin start -all and pmm-admin list output and the check network again :

omg@pc-0725:~ sudo pmm-admin start --all OK, 2 services are started. omg&#64;pc-0725:~ sudo pmm-admin list
pmm-admin 1.1.1

PMM Server | 172.16.18.102
Client Name | pc-0725
Client Address | 172.16.18.63
Service Manager | linux-systemd

SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS

linux:metrics pc-0725 42000 YES -
mongodb:metrics pc-0725 42003 YES localhost:27017

omg@pc-0725:~$ sudo pmm-admin check-network
PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

  • System Time
    Server | 2017-03-14 13:25:43 +0000 GMT
    Client | 2017-03-14 13:46:14 +0800 CST
    Time Drift | 27569s

Time is out of sync. Please make sure the server time is correct to see the metrics.

  • Connection: Client --> Server

SERVER SERVICE STATUS

Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 2.364165ms
Request duration | 1.428277ms
Full round trip | 3.792442ms

  • Connection: Client <-- Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD

linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: http://172.16.18.102/prometheus/targets

omg@pc-0725:~$

  1. I had closed the firewall on the VirtualBox PC where the PMM Server located !!

And here I also attached the URL of http://172.16.18.102/prometheus/targets output for your reference :slight_smile:

1281×1021 113 KB

#4

Do you have any firewall on MongoDB side?

#5

The MongoDB is a Linux running Ubuntu Trusty(14.04) on our LAN, and I am pretty sure there is NO firewall of any kind on it :slight_smile:

In the http://172.16.18.102/prometheus/targets ) and find a lot of data in them, so I guess the 2 ports are open !!

But the Endpoint links of http://localehost:9100/metrics for linux and the Endpoint link of http://localhost:9090/prometheus/metrics for prometheus can NOT be reached :confused:

#6

it is strange why shows DOWN, but according to targets page everything is working fine

do you have any other errors anywhere?
do you see MongoDB graphs in grafana?

#7

In the /var/log/pmm-linux-metrics-42000.log and /var/log/pmm-mongodb-metrics-42003.log, there are some errors about TLS handshake error …

omg@pc-0725:~$ cat /var/log/pmm-linux-metrics-42000.log
time=“2017-03-14T10:44:41+08:00” level=info msg=“Starting node_exporter (version=1.1.1, branch=master, revision=2d78e22000779d63c714011e4fb30c65623b9c77)” source=“node_exporter.go:170”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Build context (go=go1.7.4, user=, date=)” source=“node_exporter.go:171”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Enabled collectors:” source=“node_exporter.go:190”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - filefd" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - loadavg" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - meminfo" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - stat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - vmstat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - diskstats" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - filesystem" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - netdev" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - netstat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - time" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - uname" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=“HTTPS/TLS is enabled” source=“node_exporter.go:235”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Listening on 172.16.18.63:42000” source=“node_exporter.go:238”
2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:55114: tls: first record does not look like a TLS handshake
2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:57094: tls: first record does not look like a TLS handshake
2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:40756: tls: first record does not look like a TLS handshake
2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:54292: tls: first record does not look like a TLS handshake
2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:54728: tls: first record does not look like a TLS handshake
2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:57892: tls: first record does not look like a TLS handshake
2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:38342: tls: first record does not look like a TLS handshake

omg@pc-0725:~$ cat /var/log/pmm-mongodb-metrics-42003.log

Warning: the exporter is in beta/experimental state and field names are very

likely to change in the future and features may change or get removed!

See: https://github.com/percona/mongodb_exporter for updates

mongodb_exporter version: 1.1.1, git commit hash: c890996b9133a3288530da4a9fb6e6fad067b55f
HTTPS/TLS is enabled
Listening on 172.16.18.63:42003
2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:59072: tls: first record does not look like a TLS handshake
2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:32820: tls: first record does not look like a TLS handshake
2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:44714: tls: first record does not look like a TLS handshake
2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:58246: tls: first record does not look like a TLS handshake
2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:58686: tls: first record does not look like a TLS handshake
2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:33618: tls: first record does not look like a TLS handshake
2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:42300: tls: first record does not look like a TLS handshake
omg@pc-0725:~$

#8

can you share output of the following command?

docker ps | grep pmm-server
#9

Oh, sorry.
ssh user is ‘admin’ (we are adding this to documentation in PMM-650)

can you ssh to VirtualBox and synchronize time?
and reboot…

#10

What is the password ?

#11

immediately after the first virtualbox instance start, web interface asks username, password and public ssh key.
did you configured ssh key?

#12

" on the screen) !!

Can you show me how to configure SSH Key ?

First input “admin” for the Username, then …

#13

Do you have ssh key?

we have two options:

  1. remove current instance and create new
  2. set ssh key via REST API (command line)
#14

public ssh key should looks like one long line which starts with “ssh-rsa”

curl -X POST -d '{"Key": "ssh-rsa log-log-line some-comment"}' http://login:password&#64;PMM-SERVER-IP/configurator/v1/sshkey

as you can see, you need the current http credentials

this command should return ssh key fingerprint as response

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright © 2006 - 2021 Percona LLC. All rights reserved.