Hello.
I am starting using PMM Alerting with provided templates. It looks very cool and easy to setup, thanks a lot.
For now I have few alerts based on templates and they are working in dev environment perfectly. Now I want to backup those alerts and in case of replacing PMM server I want to restore them. Is there some mechanism to import/export alert rules and notification channels? Ideally will be to have some possibility to automate it.
I saw “Admin” in Alerting tab with some json configuration. But it’s only about notification channels.
Thanks in advance.
There isn’t a specific Alerting backup/restore mechanism. There are a few ways to do backups/restores in PMM including the recommended steps you can find here and here. The only issue I have with that approach is it requires the PMM DB’s to be quiesced (PMM server is offline) to get a physical backup and I don’ t like stopping things to take backups and you have to be running a current version of PMM (2.38.0 or newer) for pmm-client to stay up and cache metrics on each monitored node when PMM server is down so I write and maintain this tool that does a physical backup of PMM server keeping PMM online (it will stop all services to do the restore). Online backups aren’t perfect compared to offline as they 3 main db’s in PMM (postgres, victoriametrics, and clickhouse) are backed up sequentially and data will be changing for the other 2 DB’s while the third is being backed up so you’ll be consistent from an individual db standpoint but my utility doesn’t take a time-consistent backup across all 3 db’s.
The alerting templates and configured alerts are stored in the internal postgres db of PMM so if you don’t care about anything else (query data or db/node data) you can modify my utility to just backup/restore what you want.
1 Like