Mongodb is Down

Hello Team

I am install pmm client on our db server and my mongodb is running fine but in admin list coming as status down.

On firewall level we have opened the port as well.

could you please suggest ant solution here.

[xxxxx@rtls-replica-db01 ~]$ sudo pmm-admin check-network
Warning: It is recommended to use the same version on both PMM Server and Client, otherwise some features will not work correctly.
Please upgrade your PMM Client by following the instructions from
PMM Network Status

Server Address | 10.159.22.218
Client Address | 10.159.24.100 (192.168.1.4)

  • System Time
    NTP Server (0.pool.ntp.org) | unable to get ntp time: %!s()
    PMM Server | 2021-05-12 09:09:30 +0000 GMT
    PMM Client | 2021-05-12 14:39:42 +0530 IST
    PMM Client to PMM Server Time Drift | OK

  • Connection: Client → Server


SERVER SERVICE STATUS


Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 1.120987ms
Request duration | 2.695585ms
Full round trip | 3.816572ms

  • Connection: Client ← Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD


linux:metrics rtls-replica-db01.novalocal 10.159.24.100–>192.168.1.4:42000 OK YES -
mongodb:metrics rtls-replica-db01.novalocal 10.159.24.100–>192.168.1.4:42003 DOWN - -

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: http://10.159.22.218/prometheus/targets

IMPORTANT: client and bind addresses are not the same which means you need to configure NAT/port forwarding to map them.

[xxxxxx@rtls-replica-db01 ~]$

[xxxxxx@rtls-replica-db01 ~]$ sudo pmm-admin list
Warning: It is recommended to use the same version on both PMM Server and Client, otherwise some features will not work correctly.
Please upgrade your PMM Client by following the instructions from
pmm-admin 1.17.1

PMM Server | 10.159.22.218
Client Name | rtls-replica-db01.novalocal
Client Address | 10.159.24.100 (192.168.1.4)
Service Manager | linux-systemd


SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS


linux:metrics rtls-replica-db01.novalocal 42000 YES -
mongodb:metrics rtls-replica-db01.novalocal 42003 NO rtls:***@10.159.24.100:27017/admin cluster=replicaset01

[xxxxx@rtls-replica-db01 ~]$ ps -ef | grep mongo
jioapp 2884 27100 0 15:14 pts/3 00:00:00 grep --color=auto mongo
jioapp 11138 1 0 Apr08 ? 06:52:51 mongod --config=/data/mongodb-linux-x86_64-rhel70-4.4.1/config/mongodb.conf --fork

[xxxxxx@rtls-replica-db01 ~]$ sudo netstat -tulnp | grep 27017
tcp 0 0 192.168.1.4:27017 0.0.0.0:* LISTEN 11138/mongod
[xxxxxxx@rtls-replica-db01 ~]$

[xxxxx@rtls-replica-db01 ~]$ ps -ef | grep pmm
root 20942 1 0 Apr12 ? 00:00:00 /bin/sh -c /usr/local/percona/pmm-client/node_exporter -web.listen-address=192.168.1.4:42000 -web.auth-file=/usr/local/percona/pmm-client/pmm.yml -web.ssl-key-file=/usr/local/percona/pmm-client/server.key -web.ssl-cert-file=/usr/local/percona/pmm-client/server.crt -collectors.enabled=diskstats,filefd,filesystem,loadavg,meminfo,netdev,netstat,stat,time,uname,vmstat,meminfo_numa,textfile >> /var/log/pmm-linux-metrics-42000.log 2>&1
root 20943 20942 0 Apr12 ? 00:00:59 /usr/local/percona/pmm-client/node_exporter -web.listen-address=192.168.1.4:42000 -web.auth-file=/usr/local/percona/pm -client/pmm.yml -web.ssl-key-file=/usr/local/percona/pmm-client/server.key -web.ssl-cert-file=/usr/local/percona/pmm-client/server.crt -collectors.enabled=diskstats,filefd,filesystem,loadavg,meminfo,netdev,netstat,stat,time,uname,vmstat,meminfo_numa,textfile
[xxxxx@rtls-replica-db01 ~]$

Please help us.

Regards
Pranita

Hi Pranita,

Could you inform us what exact version do you use?
Is PMM1 selected in purpose?

1 Like

What version of pmm server did you install? It looks like you may be running PMM 2.x server but pmm 1.17.1 client. If so you can upgrade the client by installing pmm2-client with your favorite package manager.

1 Like

Hello Team

on pmm admin server we have below version:

[root@4449d09d106c opt]# pmm-admin --version
1.17.4
[root@4449d09d106c opt]#

pmm-client:

[xxxx@rtls-replica-db01 ~]$ sudo yum info pmm-client
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Installed Packages
Name : pmm-client
Arch : x86_64
Version : 1.17.1
Release : 1.el7
Size : 74 M
Repo : installed
Summary : Percona Monitoring and Management Client
URL : https://percona.com
License : AGPLv3
Description : Percona Monitoring and Management (PMM) is an open-source platform for managing and monitoring MySQL and MongoDB
: performance. It is developed by Percona in collaboration with experts in the field of managed database services,
: support and consulting.
: PMM is a free and open-source solution that you can run in your own environment for maximum security and reliability.
: It provides thorough time-based analysis for MySQL and MongoDB servers to ensure that your data works as efficiently
: as possible.

let us know still upgrade is required on client side or issue is something else because other databases running fine on pmm monitoring.

Awaiting for your response.

Regards
Pranita Singh

1 Like