Hi Team,
Greetings !
I am facing issue depicted in below snapshot frequently on PMM.
Please guide me to fix this issue.
Thank you in advance.
Warm Regards,
Jyoti Rajai
FJYI : I have already tired solution provided on this link: HTTP Error Bad Gateway when using prometheus · Issue #14629 · grafana/grafana · GitHub
But nothing helps.
1 Like
Hi @Jyoti_Rajai
What are the system requirements.?
- Is CPU and RAM are doing good?
- Check your internet, due to the internet slowness we may get this issue.
- Did you restarted the pmm server?
- Check the pmm2 server status weather it is healthy or not? docker ps -a
1 Like
Hi Jyoti,
Could you check if it’s enough disk space in the container?
2 Likes
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0d3c3316ff30 percona/pmm-server:latest “/opt/entrypoint.sh” 3 weeks ago 3 weeks ago 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp pmm-server
9fc459a9fdd6 percona/pmm-server:latest “/bin/true” 3 weeks ago pmm-data
1 Like
Could you check if it’s enough disk space in the container? – Yeah, 400+gb free.
1 Like
Hi @Jyoti_Rajai
Seems everything looks good.
Can you restart the PMM server and try once?
docker stop 0d3c3316ff30
docker stop 9fc459a9fdd6
systemctl restart docker
1 Like
docker stop 0d3c3316ff30
0d3c3316ff30
docker stop 9fc459a9fdd6
9fc459a9fdd6
systemctl restart docker
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0d3c3316ff30 percona/pmm-server:latest “/opt/entrypoint.sh” 3 weeks ago Up 9 seconds 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp pmm-server
9fc459a9fdd6 percona/pmm-server:latest “/bin/true” 3 weeks ago pmm-data
[root@testhodb4node3 ~]#
1 Like
Hi @Jyoti_Rajai
Can you please re-check the PMM serer status, weather it is healthy or not?
1 Like
Hi @Jyoti_Rajai
After restarting the PMM server, is it healthy or not?
docker ps -a
and, can you try to clear the cache or try it in different browser?
1 Like
Didn’t helped.
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0d3c3316ff30 percona/pmm-server:latest “/opt/entrypoint.sh” 3 weeks ago Up 9 seconds 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp pmm-server
9fc459a9fdd6 percona/pmm-server:latest “/bin/true” 3 weeks ago pmm-data
1 Like
Hi @Jyoti_Rajai
For me it is showing healthy, but for you its not showing anything either its health/unhealthy in the docker status.
and, can you please check the PMM server container log file.
docker logs --tail=100 be330f386f3f
docker logs --tail=100
1 Like
[root@testhodb4node3 ~]# docker logs --tail=20 f6b1ae985878
2022-02-09 12:53:57,802 INFO exited: postgresql (exit status 1; not expected)
2022-02-09 12:54:04,359 INFO spawned: ‘postgresql’ with pid 11516
2022-02-09 12:54:04,372 INFO exited: postgresql (exit status 1; not expected)
2022-02-09 12:54:11,803 INFO spawned: ‘postgresql’ with pid 11517
2022-02-09 12:54:11,816 INFO exited: postgresql (exit status 1; not expected)
2022-02-09 12:54:20,669 INFO spawned: ‘postgresql’ with pid 11518
2022-02-09 12:54:20,680 INFO exited: postgresql (exit status 1; not expected)
2022-02-09 12:54:29,819 INFO spawned: ‘postgresql’ with pid 11519
2022-02-09 12:54:29,829 INFO exited: postgresql (exit status 1; not expected)
2022-02-09 12:54:40,830 INFO spawned: ‘postgresql’ with pid 11520
2022-02-09 12:54:40,840 INFO exited: postgresql (exit status 1; not expected)
2022-02-09 12:54:41,830 INFO gave up: postgresql entered FATAL state, too many start retries too quickly
2022-02-10 00:01:01,705 INFO reaped unknown pid 11722
2022-02-10 01:01:01,735 INFO reaped unknown pid 11741
2022-02-10 02:01:01,764 INFO reaped unknown pid 11760
2022-02-10 03:32:01,806 INFO reaped unknown pid 11779
2022-02-11 00:01:01,447 INFO reaped unknown pid 12178
2022-02-11 01:01:01,476 INFO reaped unknown pid 12197
2022-02-11 02:01:01,509 INFO reaped unknown pid 12216
2022-02-11 03:33:01,549 INFO reaped unknown pid 12235
1 Like
Hi @Jyoti_Rajai
Seems postgresql DB has some issue, it did not start properly. Can you download the server Diagnostics and check the details. It may help this issue.
Can you stop the docker service and restart the Linux server node then try once.
1 Like
I am unable to load the page you shown in screenshot.
1 Like
Hi @Jyoti_Rajai
Can you do the below steps ?
-
docker stop
-
stop the docker service
-
Reboot the Linux server node then try once.?
1 Like
Why to reboot linux server ? That doesn’t sounds proper.
Additionally, what i have found is the docker stores all data in memory instead of given path for volume.
1 Like
@Jyoti_Rajai I am also facing similar kind of issue. PMM server is working fine and but grafana is not showing up.
1 Like