High CPU load

Percona Monitoring and Management (PMM) PMM 1.x
1

Hi,

We are seeing high CPU load on a Docker install of PMM latest and the PMM web UI is somewhat sluggish.

I have searched the forum and METRICS_MONITOR is mentioned on cpu load. Right now we don’t have METRICS_MONITOR set since memory usage seems to be ok. PMM client network check looks ok.

Currently monitoring 38 RDS nodes, but planning to add a further 60 nodes. So scaling is a concern and slowness on the PMM web interface.

We have the pmm server and client installed on the same VM. This is the first time we have tried this setup, so that may be an issue.

Some details

  • Running on a VM that is dedicated to pmm only. 6 CPU at this time and 15G ram.

  • mainly monitoring RDS instances, so using Performance schema and innodb_monitor_enable=all

  • have pmm client v1.9.1 and server v1.8.1. Note we have the server and client running on the same VM.

  • prometheus is using the most CPU

PMM server container settings

sudo docker run -d
-p 80:80
-e METRICS_RESOLUTION=5s
-e METRICS_RETENTION=720h
-e QUERIES_RETENTION=20
–volumes-from pmm-data
–name pmm-server
–restart always
percona/pmm-server:latest

Have attached files showing

  • tail prometheus log
  • top output
  • sar output
  • pmm client network check

Thanks

tail_prometheus_log.txt (2.29 KB)

top_output.jpg

sar_ouput_cpu_load.txt (651 Bytes)

pmm_client_network_check.txt (792 Bytes)

2

Since I am on pmm version 1.8.1, it looks like I do not have to set the METRICS_MONITOR to control memory usage, since PMM now does memory scaling automatically.

https://www.percona.com/blog/2018/02/27/percona-monitoring-management-1-8-0-now-available/