your consumption sounds really high…I’m monitoring 10 nodes and using ~750MB on my pmm-server container, our pmmdemo site is monitoring 30 nodes and only using about 4.2GB. We have a general recommendation here on sizing but it’s definitely padded to be safe.
I’m curious to know what’s consuming all the memory, could you do the following and let us know:
get a bash prompt inside the docker container: docker exec -it pmm-server bash
and see what the main consumer of memory is: ps -aux
That would at least give a place to start checking logs for something running haywire.
My immediate questions would be around: scrape interval: do you have things collecting at the highest resolution? Have you installed any additional grafana plugins? How did you transition to PMM2 from PMM1: whole new server, sid-by-side. What is your retention period and how active are users in consuming the graphs?
Those are all the things I can think of off the top of my head that would be the big offenders. VictoriaMetrics will certainly reduce the footprint but sounds like there’s something else going wrong.