Apologies if this is more of a question about how Prometheus works, and less about PMM but here’s what I have:
After running the standard setup commands from the documentation for PMMv2:
sudo yum install https://repo.percona.com/yum/percona-release-latest.noarch.rpm sudo yum install pmm2-client pmm-admin config --server-insecure-tls --server-url=https://myPMMserver pmm-admin add mysql --username=user --password=pw --custom-labels=“environment=DEV,database=pricing”
When I query the metrics in Prometheus, I see that the MYSQL metrics contain the custom labels I added, while metrics that come directly from the node (f.e. memory or CPU) do not contain those custom labels. This is important to me because I want to feed alerts from these metrics into Alertmanager which can use the labels to route differently (f.e. alerts on prod servers go to DBA pagers, alerts on non-prod go to e-mail).
Is there a way I can add the custom labels in at the node level as well as the mysql service level? (by looking at it maybe at the pmm-admin config command? but I don’t have the ability to easily test it)