MongoDB PMM Installation

Hello, I am trying to install PMM tool to monitor a MongoDB client in the same network, the MongoDB client is on a ubuntu trusty(so I am using pmm-client_1.1.1-1.trusty_amd64.deb to install), and the PMM Server is on a VirtualBox(so I am using PMM-Server-2017-02-16.ova to monitor) !! But after I install them, I can NOT see any graph on the PMM Server … here is the “pmm-admin check-network” output :

PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

  • System Time
    Server | 2017-03-14 11:30:28 +0000 GMT
    Client | 2017-03-14 11:30:52 +0800 CST
    Time Drift | 28776s

Time is out of sync. Please make sure the server time is correct to see the metrics.

  • Connection: Client → Server

SERVER SERVICE STATUS


Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 861.946µs
Request duration | 4.927953ms
Full round trip | 5.789899ms

  • Connection: Client ← Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD


linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: [url]http://172.16.18.102/prometheus/targets[/url]

It seems that the service status is DOWN … what should I do ? Can you help me ?

Hi bobho,

  1. can you run and check network again?
    also please run command

  2. do you have any kind of firewall?
    it is needed to allow connection from pmm-server to pmm-client 42000, 42003 ports.

Hi! Mykola :

Here is the pmm-admin start -all and pmm-admin list output and the check network again :

omg@pc-0725:~$ sudo pmm-admin start --all
OK, 2 services are started.
omg@pc-0725:~$ sudo pmm-admin list
pmm-admin 1.1.1

PMM Server | 172.16.18.102
Client Name | pc-0725
Client Address | 172.16.18.63
Service Manager | linux-systemd


SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS


linux:metrics pc-0725 42000 YES -
mongodb:metrics pc-0725 42003 YES localhost:27017

omg@pc-0725:~$ sudo pmm-admin check-network
PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

  • System Time
    Server | 2017-03-14 13:25:43 +0000 GMT
    Client | 2017-03-14 13:46:14 +0800 CST
    Time Drift | 27569s

Time is out of sync. Please make sure the server time is correct to see the metrics.

  • Connection: Client → Server

SERVER SERVICE STATUS


Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 2.364165ms
Request duration | 1.428277ms
Full round trip | 3.792442ms

  • Connection: Client ← Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD


linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: [url]http://172.16.18.102/prometheus/targets[/url]

omg@pc-0725:~$

  1. I had closed the firewall on the VirtualBox PC where the PMM Server located !!

And here I also attached the URL of [url]http://172.16.18.102/prometheus/targets[/url] output for your reference :slight_smile:

Do you have any firewall on MongoDB side?

The MongoDB is a Linux running Ubuntu Trusty(14.04) on our LAN, and I am pretty sure there is NO firewall of any kind on it :slight_smile:

In the [URL]http://172.16.18.102/prometheus/targets[/URL] ) and find a lot of data in them, so I guess the 2 ports are open !!

But the Endpoint links of [url]http://localehost:9100/metrics[/url] for linux and the Endpoint link of [url]http://localhost:9090/prometheus/metrics[/url] for prometheus can NOT be reached :confused:

it is strange why shows DOWN, but according to targets page everything is working fine

do you have any other errors anywhere?
do you see MongoDB graphs in grafana?

In the /var/log/pmm-linux-metrics-42000.log and /var/log/pmm-mongodb-metrics-42003.log, there are some errors about TLS handshake error …

omg@pc-0725:~$ cat /var/log/pmm-linux-metrics-42000.log
time=“2017-03-14T10:44:41+08:00” level=info msg=“Starting node_exporter (version=1.1.1, branch=master, revision=2d78e22000779d63c714011e4fb30c65623b9c77)” source=“node_exporter.go:170”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Build context (go=go1.7.4, user=, date=)” source=“node_exporter.go:171”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Enabled collectors:” source=“node_exporter.go:190”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - filefd" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - loadavg" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - meminfo" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - stat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - vmstat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - diskstats" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - filesystem" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - netdev" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - netstat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - time" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - uname" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=“HTTPS/TLS is enabled” source=“node_exporter.go:235”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Listening on 172.16.18.63:42000” source=“node_exporter.go:238”
2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:55114: tls: first record does not look like a TLS handshake
2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:57094: tls: first record does not look like a TLS handshake
2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:40756: tls: first record does not look like a TLS handshake
2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:54292: tls: first record does not look like a TLS handshake
2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:54728: tls: first record does not look like a TLS handshake
2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:57892: tls: first record does not look like a TLS handshake
2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:38342: tls: first record does not look like a TLS handshake

omg@pc-0725:~$ cat /var/log/pmm-mongodb-metrics-42003.log

Warning: the exporter is in beta/experimental state and field names are very

likely to change in the future and features may change or get removed!

See: [url]https://github.com/percona/mongodb_exporter[/url] for updates

mongodb_exporter version: 1.1.1, git commit hash: c890996b9133a3288530da4a9fb6e6fad067b55f
HTTPS/TLS is enabled
Listening on 172.16.18.63:42003
2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:59072: tls: first record does not look like a TLS handshake
2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:32820: tls: first record does not look like a TLS handshake
2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:44714: tls: first record does not look like a TLS handshake
2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:58246: tls: first record does not look like a TLS handshake
2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:58686: tls: first record does not look like a TLS handshake
2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:33618: tls: first record does not look like a TLS handshake
2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:42300: tls: first record does not look like a TLS handshake
omg@pc-0725:~$

can you share output of the following command?

docker ps | grep pmm-server

Oh, sorry.
ssh user is ‘admin’ (we are adding this to documentation in PMM-650)

can you ssh to VirtualBox and synchronize time?
and reboot…

What is the password ?

immediately after the first virtualbox instance start, web interface asks username, password and public ssh key.
did you configured ssh key?

" on the screen) !!

Can you show me how to configure SSH Key ?

First input “admin” for the Username, then …

Do you have ssh key?

we have two options:

  1. remove current instance and create new
  2. set ssh key via REST API (command line)

public ssh key should looks like one long line which starts with “ssh-rsa”

curl -X POST -d '{"Key": "ssh-rsa log-log-line some-comment"}' http://login:password@PMM-SERVER-IP/configurator/v1/sshkey

as you can see, you need the current http credentials

this command should return ssh key fingerprint as response