Description:
I am creating an alert to tell me if an instance in my MySQL Cluster has a Group Replication Node OFFLINE. I create the rule and it fires fine and is alerting. However, it is not getting pushed to the Notification Policy to alert me via Slack because the Labels the alert generates are more than what I have set for the Notification Policy. I don’t know why that is happening. It seems the labels it generates matches the columns returned from the Preview Alert button while editing the rule. I don’t know how to control that.
Steps to Reproduce:
Notification Policy Set to alert on 2 labels:severity=critical and grafana_folder=MySQL Non-Production.
For the rule, the Metric expression used is mysql_perf_schema_replication_group_member_info{service_name="ia-se-vudb3-123-mysql", member_state="OFFLINE"}
The rule is being stored in Folder MySQL Non-Production.
The Group is default-alert-group.
The notifications has two labels for it: severity=critical and grafana_folder=MySQL Non-Production.
Also when viewing the rule, the Labels display several additional labels which were automatically created: alert name, datasource_uid, red_id. This may have a factor on why there are so many Labels when the Alert is firing and Alerting. I’ll provide screen shot.
Version:
2.38.1
Any help would be appreciated.
Thanks
DD
