forgive my belated response - didn’t realize e-mail notifications for forum post are set to “off”.
lorraine.pocklington I’ve seen/read the posts, but admittedly not every little bit of each post - simply thought that data clean-up should work as simple as Mykola suggested in this post: https://www.percona.com/forums/questions-discussions/percona-monitoring-and-management/48664-how-to-clean-up-space-without-effect-the-config
Few probably important things I forgot to mention:
- PMM is monitoring 4 servers in total: 3 Galera cluster nodes, and a replication slave (That shouldn’t produce too much data, right? Even in spite of the point below)
- The settings are default, except for: innodb_monitor_enable=‘all’ and log_slow_verbosity=‘full’
- There’s a custom dashboard with alerts configured, twelve of them to be exact,… and all except two (the DDL statements count) are generic.
Back when clean-up was intended by reducing the metrics retention to 3, 2, and then 1 month, versions were 1.6.1, and 1.7.0. Funny thing is that the clean-up
worked on the staging PMM instance: 6h after I reduced the QUERY/METRICS_RETENTION to 720h, nearly 30GB was freed up on an 80GB disk.
With that in mind, it just didn’t make sense for this not to work for the production server: I keep shortening the retention period, while disk usage by PMM keeps growing. And I’m sure it’s not something else eating up the space, like a rogue log file… So maybe it’s how much data I’m collecting by using settings above.
Anyhow, my storage space problem is (temporarily?) solved - I’ve resorted to painfully slow process of moving Docker data to the add-on volume (at 30MB/s), to see how it works after. The IOPS performance of the volume, which I initially overlooked, was indeed that of an SSD, although capped to ~3k,… just like the (sequential) throughput is.