# How to invesigate integrated alert not show a mysql was done **URL:** \ **Category:** PMM 2.x\ **Created:** [February 13, 2023, 1:14am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044 "2023-02-13T01:14:34Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Dillonlu](https://avatars.discourse-cdn.com/v4/letter/d/ce73a5/32.png) [@Dillonlu](https://forums.percona.com/u/Dillonlu)\ **Post date:** [February 13, 2023, 1:14am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/1 "2023-02-13T01:14:34Z")
Hi All, Last Friday, 2/10, my one mysql instance was down, but the integrated alert wasn’t shown any error message. By running the select statement in explore window, I saw the mysql service was done a few miniutes. Please see attached screenshot. The reason was the VM node crashed. The VM was transfered to another node. My question is how I investigate why the alert is failed to show in PMM. I’m not show which component cause this problem. Agent? Alertmanager? or Email system? Thank, Dillon ![screenshot](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/e/e830bf83a5173a624841034b20722490d4bbc339.png) ---
**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\ **Post date:** [February 13, 2023, 12:35pm UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/2 "2023-02-13T12:35:17Z")
Hi, the alert won’t fire because mysql\_up metric is not 0 but simply not present for some time. You might want to experiment with the absent\_over\_time function for alerting. ---
**Author:** ![Dillonlu](https://avatars.discourse-cdn.com/v4/letter/d/ce73a5/32.png) [@Dillonlu](https://forums.percona.com/u/Dillonlu)\ **Post date:** [February 14, 2023, 1:47am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/3 "2023-02-14T01:47:23Z")
Hi Ivan, Thank you for suggestion. Thanks, Dillon ---
**Author:** ![Dillonlu](https://avatars.discourse-cdn.com/v4/letter/d/ce73a5/32.png) [@Dillonlu](https://forums.percona.com/u/Dillonlu)\ **Post date:** [February 14, 2023, 9:44am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/4 "2023-02-14T09:44:05Z")
Hi Ivan, I have to confirm a question again. The absent\_over\_time works. The explore page shows the correct result. However, the alert rule isn’t fired. My expression of the alert rule is “sum by (service\_name,node\_name) (mysql\_up) ==0 or absent\_over\_time(sum by (service\_name,node\_name) (mysql\_up))==1” Could you please give me some advise again? How to investigate? Thanks, Dillon ---
**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\ **Post date:** [February 14, 2023, 11:37am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/5 "2023-02-14T11:37:31Z")
Hi Dillon, what about something like this? `sum by (service_name,node_name) (mysql_up) ==0 or sum by (service_name,node_name) (absent_over_time(mysql_up))==1` ---
**Author:** ![Dillonlu](https://avatars.discourse-cdn.com/v4/letter/d/ce73a5/32.png) [@Dillonlu](https://forums.percona.com/u/Dillonlu)\ **Post date:** [February 15, 2023, 9:42am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/6 "2023-02-15T09:42:06Z")
Hello Ivan, Thank you for help again. Your query can return metrics data in explore. However, the same problem, the alert rule isn’t fired. I attached two screenshots. One is alert rule; the other is explore page. I hope I miss something. ![2](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/0320c4adac6de110318a383bbc74777c724b16d4.png) On above picture, I indicate the node suffix “115” I expect to show up. ![1](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/3/380956925776b74c4adbb295369546881bf4fa24.png) Thanks, Dillon ---
**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\ **Post date:** [February 15, 2023, 12:41pm UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/7 "2023-02-15T12:41:59Z")
Hi Dillon, the built-in alert only catches the case were mysql is down but the pmm-agent is still up and reporting metrics to the server. I was expecting the absent\_over\_time would catch the case were server is down but looks like it won’t work. This issue was already raised to the dev team so I suggest you vote on it: [[PMM-9544] Alerting doesn't seem to be able to monitor for "Host Down" - Percona JIRA](https://jira.percona.com/browse/PMM-9544) Regards Ivan ---
**Author:** ![Dillonlu](https://avatars.discourse-cdn.com/v4/letter/d/ce73a5/32.png) [@Dillonlu](https://forums.percona.com/u/Dillonlu)\ **Post date:** [February 17, 2023, 1:57am UTC](https://forums.percona.com/t/how-to-invesigate-integrated-alert-not-show-a-mysql-was-done/20044/8 "2023-02-17T01:57:37Z")
> [@Ivan\_Groenewold](#): > > vote sure. Thanks, Dillon