After enabling the Integrated Alerting feature, no alert messages are received.
How can I find out the reason?
Hi @peng19832
Did you add any filters in the Alert rules section?
2 Likes
Other things to check:
did you add the configured webhook notification channel to the alert? (it’s not required and means it will only display on the alerts tab)
you can download diagnostic data (gear icon → Settings → Download Server Diagnostics
and in the zip file are all the logs…look at vmalert.log, alertmanager.log, and pmm-managed.log to see if there are any errors about calling the webhook (maybe a connection refused or invalid credentials). Have seen more than one instance where a corporate firewall/proxy rule blocked PMM server from connecting to the outside world.
2 Likes
Hello, steve
I upload the log file.
Firewall is turned off.
vmalert.log:
vmalert.log (140.7 KB)
alertmanager.log:
alertmanager.log (27.8 KB)
pmm-managed.log:
pmm-managed.log (200.4 KB)
1 Like
In addition, the alerts I configured on the panel are normal.
1 Like
Hi, like Steve mentioned, to receive messages you need to configure a destination in Settings → Communication. Check out this post for more details: MongoDB Integrated Alerting in Percona Monitoring and Management - Percona Database Performance Blog
1 Like
Hi,igroene,thks for reply.
Why do I need a Communication when I define Notification Channels? Communication cannot be applied to Ali Dingding.
1 Like
I am sorry I misunderstood what you are trying to do here, please ignore my previous reply. The most likely cause is that communication from PMM to the web hook is not working properly. Anything in the logs about connection timeout/refuse?
1 Like
It s OK.
The log file mentioned by Steve, which I have uploaded in the reply above.
I don’t see any useful exceptions in the logs, or maybe I don’t know enough about the PMM logs, causing me to ignore useful information?
Can you help me analyze it, thanks?
1 Like
I meet same problem, my browser debug network info，I got the request format which dont meet the requirement of lark group robot webhook. So I startup a proxy hosted one python script for rewrite request and sent it to webhook address, then it works! Attach method below:
好的，可以只开启 Flask 的 debug 模式，并在处理请求的过程中打印实际的告警信息。下面是简化后的脚本：
from flask import Flask, request, jsonify
import requests
app = Flask(__name__)
@app.route('/webhook', methods=['POST'])
def webhook():
data = request.json
# 打印接收到的请求数据
print("Received data:", data)
# 提取和格式化实际的告警信息
if data and 'alerts' in data:
alerts = data['alerts']
for alert in alerts:
status = alert.get('status', 'Unknown status')
labels = alert.get('labels', {})
annotations = alert.get('annotations', {})
instance = labels.get('instance', 'Unknown instance')
summary = annotations.get('summary', 'No summary')
description = annotations.get('description', 'No description')
# 格式化为飞书机器人的消息格式
new_data = {
"msg_type": "text",
"content": {
"text": f"告警状态: {status}\n实例: {instance}\n摘要: {summary}\n详细信息: {description}"
}
}
# 转发请求到飞书 Webhook URL
feishu_webhook_url = 'https://open.feishu.cn/open-apis/bot/v2/hook/your-webhook-id'
response = requests.post(feishu_webhook_url, json=new_data)
# 打印转发结果
print("Sent to Feishu:", new_data)
print("Feishu response status:", response.status_code, "response:", response.json())
return jsonify({"status": "received"}), 200
if __name__ == '__main__':
app.run(host='0.0.0.0', port=8080, debug=True)
运行脚本
保存脚本到一个文件（例如 webhook_server.py），然后运行脚本：
python webhook_server.py
说明
-
开启 Flask 的 debug 模式：
app.run(host='0.0.0.0', port=8080, debug=True)这行代码启用了 Flask 的 debug 模式，使得 Flask 会打印更多的调试信息。 -
打印接收到的请求数据：
print("Received data:", data)这行代码会在终端打印接收到的请求数据。 -
提取和格式化告警信息：
根据接收到的告警信息，提取并格式化成飞书机器人要求的格式。 -
转发请求到飞书 Webhook URL：
使用requests.post方法将格式化的消息发送到飞书机器人 Webhook URL。 -
打印转发结果：
print("Sent to Feishu:", new_data)和print("Feishu response status:", response.status_code, "response:", response.json())这些代码会在终端打印发送到飞书的消息内容和飞书的响应状态、响应内容。
这样，你可以在终端看到实际的告警信息，并确保这些信息被正确格式化和转发。
@Jeromeyysun 谢谢大佬的回复，我后面试试看有没有用。 