# Integrated Alerts - Is there no resolution message for when an alert returns to normal?

**URL:** <https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412>\
**Category:** PMM 2.x\
**Created:** [February 15, 2022, 6:51pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412 "2022-02-15T18:51:02Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brian\_Clark](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/brian_clark/32/6825_2.png) [@Brian\_Clark](https://forums.percona.com/u/Brian_Clark)\
**Post date:** [February 15, 2022, 6:51pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/1 "2022-02-15T18:51:02Z")

</div>

Howdy,

I’ve just gotten into setting up the new alert system. I’m testing with the “mysql down” alert rule.

So far, it works! I can get an alert into a Slack channel when I run tests by taking down the mysql service on a node.

Is there a way to receive an “alert resolved”-type of message in Slack when the service returns to normal? Is it an extra config on my end, or a new feature that will come along eventually? Not possible/planned?

Thanks,

---

<div class="post-metadata">

**Author:** ![vasyl.yurkovych](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vasyl.yurkovych/32/132_2.png) [@vasyl.yurkovych](https://forums.percona.com/u/vasyl.yurkovych)\
**Post date:** [February 16, 2022, 11:27am UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/2 "2022-02-16T11:27:06Z")

</div>

Hello. For now, there is no way for you to get a Slack notification if the alert was resolved. It can be requested as a new feature.  
So far we have the ability to notify about resolved alerts for the Webhook notification channel.

---

<div class="post-metadata">

**Author:** ![Brian\_Clark](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/brian_clark/32/6825_2.png) [@Brian\_Clark](https://forums.percona.com/u/Brian_Clark)\
**Post date:** [February 16, 2022, 2:16pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/3 "2022-02-16T14:16:30Z")

</div>

Thank you, @vasyl.yurkovych.

I’ll look to see if it isn’t already a feature request (honestly, how could it not be?) and enter one.

---

<div class="post-metadata">

**Author:** ![vasyl.yurkovych](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vasyl.yurkovych/32/132_2.png) [@vasyl.yurkovych](https://forums.percona.com/u/vasyl.yurkovych)\
**Post date:** [February 16, 2022, 4:01pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/4 "2022-02-16T16:01:30Z")

</div>

@Brian_Clark looks like it is implemented via API. You could send a POST request ‘http://{PMM\_SERVER\_IP}/v1/management/ia/Channels/Add’

with body like:  
‘{“summary”:“Slack NC”,“slack\_config”:{“channel”:“some\_channel\_name”, “send\_resolved”: true}}’

This should enable notifications for resolved alerts for this channel

---

<div class="post-metadata">

**Author:** ![vasyl.yurkovych](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vasyl.yurkovych/32/132_2.png) [@vasyl.yurkovych](https://forums.percona.com/u/vasyl.yurkovych)\
**Post date:** [February 16, 2022, 5:20pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/5 "2022-02-16T17:20:50Z")

</div>

@Brian_Clark Sorry for the confusion. But looks like we have an issue with the `send_resolved` flag. I’ve logged a [bug](https://jira.percona.com/browse/PMM-9612) for it.

---

<div class="post-metadata">

**Author:** ![Brian\_Clark](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/brian_clark/32/6825_2.png) [@Brian\_Clark](https://forums.percona.com/u/Brian_Clark)\
**Post date:** [February 16, 2022, 7:25pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/6 "2022-02-16T19:25:31Z")

</div>

Ah, that’s a bummer! Thanks for logging the bug.

While I’ve got you here, can you tell me where on the PMM server’s filesystem I can find the Integrated Alerting built-in template files? I wanted to figure out how they work so I can create my own. I’ve been searching but haven’t tracked them down yet. Or, if better, point me at a “canonical” example of template to start from?

Thanks,

---

<div class="post-metadata">

**Author:** ![Brian\_Clark](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/brian_clark/32/6825_2.png) [@Brian\_Clark](https://forums.percona.com/u/Brian_Clark)\
**Post date:** [February 16, 2022, 7:27pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/7 "2022-02-16T19:27:42Z")

</div>

@vasyl.yurkovych Sorry, forgot to @ you for this ☝ Send me somewhere else if this is not your cup of tea.

---

<div class="post-metadata">

**Author:** ![vasyl.yurkovych](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vasyl.yurkovych/32/132_2.png) [@vasyl.yurkovych](https://forums.percona.com/u/vasyl.yurkovych)\
**Post date:** [February 16, 2022, 7:50pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/8 "2022-02-16T19:50:02Z")

</div>

@Brian_Clark you’re welcome! You can find them in our [repo](https://github.com/percona/pmm-managed/tree/main/data/iatemplates). Enjoy 🙂

---

<div class="post-metadata">

**Author:** ![Brian\_Clark](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/brian_clark/32/6825_2.png) [@Brian\_Clark](https://forums.percona.com/u/Brian_Clark)\
**Post date:** [February 17, 2022, 3:52pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/9 "2022-02-17T15:52:17Z")

</div>

@vasyl.yurkovych For what it’s worth, this did work for me. I get resolved messages in Slack now 👍

---

<div class="post-metadata">

**Author:** ![Brian\_Clark](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/brian_clark/32/6825_2.png) [@Brian\_Clark](https://forums.percona.com/u/Brian_Clark)\
**Post date:** [March 23, 2022, 11:31am UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/10 "2022-03-23T11:31:40Z")

</div>

Although it is a bit spotty, in that if two nodes fire an alert I might get a resolved notice for just one of them. A work in progress, I guess 🙂

---

<div class="post-metadata">

**Author:** ![jhon.gomez](https://avatars.discourse-cdn.com/v4/letter/j/df705f/32.png) [@jhon.gomez](https://forums.percona.com/u/jhon.gomez)\
**Post date:** [May 26, 2022, 10:44pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/11 "2022-05-26T22:44:49Z")

</div>

Hello, I have the same need (send resolved alerts through the slack channel), however, when I send the POST request, it is not sent. Is it always the same url path? I have PMM version (2.27.0 (April 12, 2022))

---

<div class="post-metadata">

**Author:** ![vasyl.yurkovych](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vasyl.yurkovych/32/132_2.png) [@vasyl.yurkovych](https://forums.percona.com/u/vasyl.yurkovych)\
**Post date:** [May 27, 2022, 11:11am UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/12 "2022-05-27T11:11:39Z")

</div>

Hello @jhon.gomez ! Yes the endpoint should be the same? could you please share the url? and a response message and code?

---

<div class="post-metadata">

**Author:** ![jhon.gomez](https://avatars.discourse-cdn.com/v4/letter/j/df705f/32.png) [@jhon.gomez](https://forums.percona.com/u/jhon.gomez)\
**Post date:** [May 27, 2022, 10:31pm UTC](https://forums.percona.com/t/integrated-alerts-is-there-no-resolution-message-for-when-an-alert-returns-to-normal/14412/14 "2022-05-27T22:31:55Z")

</div>

![Screen Shot 2022-05-27 at 4.35.10 PM](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/0a26b39ec59e955a3395f103083f1f21025b37a8.png)
