MongoDB PMM Installation

bobho · March 13, 2017, 10:04pm

Hello, I am trying to install PMM tool to monitor a MongoDB client in the same network, the MongoDB client is on a ubuntu trusty(so I am using pmm-client_1.1.1-1.trusty_amd64.deb to install), and the PMM Server is on a VirtualBox(so I am using PMM-Server-2017-02-16.ova to monitor) !! But after I install them, I can NOT see any graph on the PMM Server … here is the “pmm-admin check-network” output :

PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

System Time
Server | 2017-03-14 11:30:28 +0000 GMT
Client | 2017-03-14 11:30:52 +0800 CST
Time Drift | 28776s

Time is out of sync. Please make sure the server time is correct to see the metrics.

Connection: Client → Server

SERVER SERVICE STATUS

Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 861.946µs
Request duration | 4.927953ms
Full round trip | 5.789899ms

Connection: Client ← Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD

linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: [url]http://172.16.18.102/prometheus/targets[/url]

It seems that the service status is DOWN … what should I do ? Can you help me ?

Mykola · March 14, 2017, 12:08am

Hi bobho,

can you run and check network again?
also please run command
do you have any kind of firewall?
it is needed to allow connection from pmm-server to pmm-client 42000, 42003 ports.

bobho · March 14, 2017, 12:37am

Hi! Mykola :

Here is the pmm-admin start -all and pmm-admin list output and the check network again :

omg@pc-0725:~$ sudo pmm-admin start --all
OK, 2 services are started.
omg@pc-0725:~$ sudo pmm-admin list
pmm-admin 1.1.1

PMM Server | 172.16.18.102
Client Name | pc-0725
Client Address | 172.16.18.63
Service Manager | linux-systemd

SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS

linux:metrics pc-0725 42000 YES -
mongodb:metrics pc-0725 42003 YES localhost:27017

omg@pc-0725:~$ sudo pmm-admin check-network
PMM Network Status

Server Address | 172.16.18.102
Client Address | 172.16.18.63

System Time
Server | 2017-03-14 13:25:43 +0000 GMT
Client | 2017-03-14 13:46:14 +0800 CST
Time Drift | 27569s

Time is out of sync. Please make sure the server time is correct to see the metrics.

Connection: Client → Server

SERVER SERVICE STATUS

Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 2.364165ms
Request duration | 1.428277ms
Full round trip | 3.792442ms

Connection: Client ← Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD

linux:metrics pc-0725 172.16.18.63:42000 DOWN YES -
mongodb:metrics pc-0725 172.16.18.63:42003 DOWN YES -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: [url]http://172.16.18.102/prometheus/targets[/url]

omg@pc-0725:~$

I had closed the firewall on the VirtualBox PC where the PMM Server located !!

And here I also attached the URL of [url]http://172.16.18.102/prometheus/targets[/url] output for your reference

Mykola · March 14, 2017, 1:10am

Do you have any firewall on MongoDB side?

bobho · March 14, 2017, 1:41am

The MongoDB is a Linux running Ubuntu Trusty(14.04) on our LAN, and I am pretty sure there is NO firewall of any kind on it

In the [URL]http://172.16.18.102/prometheus/targets[/URL] ) and find a lot of data in them, so I guess the 2 ports are open !!

But the Endpoint links of [url]http://localehost:9100/metrics[/url] for linux and the Endpoint link of [url]http://localhost:9090/prometheus/metrics[/url] for prometheus can NOT be reached

Mykola · March 14, 2017, 1:53am

it is strange why shows DOWN, but according to targets page everything is working fine

do you have any other errors anywhere?
do you see MongoDB graphs in grafana?

bobho · March 14, 2017, 2:03am

In the /var/log/pmm-linux-metrics-42000.log and /var/log/pmm-mongodb-metrics-42003.log, there are some errors about TLS handshake error …

omg@pc-0725:~$ cat /var/log/pmm-linux-metrics-42000.log
time=“2017-03-14T10:44:41+08:00” level=info msg=“Starting node_exporter (version=1.1.1, branch=master, revision=2d78e22000779d63c714011e4fb30c65623b9c77)” source=“node_exporter.go:170”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Build context (go=go1.7.4, user=, date=)” source=“node_exporter.go:171”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Enabled collectors:” source=“node_exporter.go:190”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - filefd" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - loadavg" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - meminfo" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - stat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - vmstat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - diskstats" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - filesystem" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - netdev" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - netstat" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - time" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=" - uname" source=“node_exporter.go:192”
time=“2017-03-14T10:44:41+08:00” level=info msg=“HTTPS/TLS is enabled” source=“node_exporter.go:235”
time=“2017-03-14T10:44:41+08:00” level=info msg=“Listening on 172.16.18.63:42000” source=“node_exporter.go:238”
2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:55114: tls: first record does not look like a TLS handshake
2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:57094: tls: first record does not look like a TLS handshake
2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:40756: tls: first record does not look like a TLS handshake
2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:54292: tls: first record does not look like a TLS handshake
2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:54728: tls: first record does not look like a TLS handshake
2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:57892: tls: first record does not look like a TLS handshake
2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:38342: tls: first record does not look like a TLS handshake

omg@pc-0725:~$ cat /var/log/pmm-mongodb-metrics-42003.log

Warning: the exporter is in beta/experimental state and field names are very

likely to change in the future and features may change or get removed!

See: [url]https://github.com/percona/mongodb_exporter[/url] for updates

mongodb_exporter version: 1.1.1, git commit hash: c890996b9133a3288530da4a9fb6e6fad067b55f
HTTPS/TLS is enabled
Listening on 172.16.18.63:42003
2017/03/14 10:44:49 http: TLS handshake error from 172.16.18.63:59072: tls: first record does not look like a TLS handshake
2017/03/14 11:30:52 http: TLS handshake error from 172.16.18.63:32820: tls: first record does not look like a TLS handshake
2017/03/14 11:48:31 http: TLS handshake error from 172.16.18.63:44714: tls: first record does not look like a TLS handshake
2017/03/14 13:40:54 http: TLS handshake error from 172.16.18.63:58246: tls: first record does not look like a TLS handshake
2017/03/14 13:41:33 http: TLS handshake error from 172.16.18.63:58686: tls: first record does not look like a TLS handshake
2017/03/14 13:46:14 http: TLS handshake error from 172.16.18.63:33618: tls: first record does not look like a TLS handshake
2017/03/14 15:25:00 http: TLS handshake error from 172.16.18.63:42300: tls: first record does not look like a TLS handshake
omg@pc-0725:~$

Mykola · March 14, 2017, 4:40am

can you share output of the following command?

docker ps | grep pmm-server

Mykola · March 15, 2017, 12:44am

Oh, sorry.
ssh user is ‘admin’ (we are adding this to documentation in PMM-650)

can you ssh to VirtualBox and synchronize time?
and reboot…

bobho · March 15, 2017, 3:11am

What is the password ?

Mykola · March 15, 2017, 3:51am

immediately after the first virtualbox instance start, web interface asks username, password and public ssh key.
did you configured ssh key?

bobho · March 15, 2017, 4:15am

" on the screen) !!

Can you show me how to configure SSH Key ?

First input “admin” for the Username, then …

Mykola · March 16, 2017, 1:55am

Do you have ssh key?

we have two options:

remove current instance and create new
set ssh key via REST API (command line)

Mykola · March 16, 2017, 5:15am

public ssh key should looks like one long line which starts with “ssh-rsa”

curl -X POST -d '{"Key": "ssh-rsa log-log-line some-comment"}' http://login:password&#64;PMM-SERVER-IP/configurator/v1/sshkey

as you can see, you need the current http credentials

this command should return ssh key fingerprint as response

Topic		Replies	Views
Mongodb is Down PMM 2.x pmm , percona , mongodb	3	479	May 13, 2021
Prometheus is down. Please check if PMM server container runs properly. PMM 1.x	0	986	September 13, 2018
PMM does not display metrics when either agent or target MongoDB is down PMM 1.x	7	1687	August 17, 2017
PMM MySql 8.0.11 Instances Not Showing Graphs PMM 1.x	28	2412	January 17, 2020
pmm-admin check-network showing incorrect data? PMM 1.x	1	1352	September 14, 2016

MongoDB PMM Installation

Also you can check the endpoint status by the URL: [url]http://172.16.18.102/prometheus/targets[/url]

Warning: the exporter is in beta/experimental state and field names are very

likely to change in the future and features may change or get removed!

See: [url]https://github.com/percona/mongodb_exporter[/url] for updates

Related topics