# \[PMM 3.1.0\] Alertmanager TLS Certificate Trust Issue with Proxy (Slack Webhook) **URL:** \ **Category:** PMM 3.x\ **Tags:** pmm\ **Created:** [May 12, 2025, 8:21am UTC](https://forums.percona.com/t/pmm-3-1-0-alertmanager-tls-certificate-trust-issue-with-proxy-slack-webhook/37906 "2025-05-12T08:21:15Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![pmm3.1](https://avatars.discourse-cdn.com/v4/letter/p/8edcca/32.png) [@pmm3.1](https://forums.percona.com/u/pmm3.1)\ **Post date:** [May 12, 2025, 8:21am UTC](https://forums.percona.com/t/pmm-3-1-0-alertmanager-tls-certificate-trust-issue-with-proxy-slack-webhook/37906/1 "2025-05-12T08:21:15Z")
I’m currently using PMM 3.1.0 (Docker) and trying to configure Alertmanager to send Slack notifications through a corporate proxy server. However, I’m facing a TLS issue due to a self-signed (or internally signed) certificate on the proxy. Goal : To allow PMM Alertmanager to successfully send Slack webhook notifications through a corporate HTTPS proxy (e.g., [proxy.example.com:3333](http://proxy.example.com:3333)) which uses a custom certificate authority (CA). What I’ve tried so far 1. insecure\_skip\_verify: true in alertmanager.base.yml ```auto receivers: - name: 'slack' slack_configs: - api_url: 'https://hooks.slack.com/services/XXX/YYYY/ZZZ' channel: '#alerts' send_resolved: true http_config: tls_config: insecure_skip_verify: true ``` → Restarted pmm-managed with supervisorctl restart pmm-managed → No effect. TLS x509 error persists. 1. Specifying CA certificate file via ca\_file Copied internally trusted proxy-ca-cert.crt to /srv/alertmanager/ Updated config: ```auto http_config: tls_config: ca_file: /srv/alertmanager/proxy-ca-cert.crt ``` → Still not applied. Alertmanager does not appear to respect this setting. 1. System-wide CA trust update attempt Placed .crt into /etc/pki/ca-trust/source/anchors/ `Ran update-ca-trust extract` → Fails with: p11-kit: couldn’t create file … Permission denied (possibly due to container’s limited root access) Questions : In PMM 3.1.0, how can we properly configure Alertmanager to: 1. skip TLS verification (like, --insecure), 2. trust a custom certificate (e.g., for proxy use)? 3. Does alertmanager.base.yml file is working in PMM 3.x alertmanager? Environment PMM version: 3.1.0 Deployment: Docker (manually run container) Outbound proxy: [proxy.example.com:3333](http://proxy.example.com:3333) (with internal CA) Error message: x509: certificate signed by unknown authority How can I handle Slack messaging through my company proxy server? Any guidance on how to proceed would be greatly appreciated. Thanks in advance for your help! ---
**Author:** ![nurlan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/nurlan/32/1523_2.png) [@nurlan](https://forums.percona.com/u/nurlan)\ **Post date:** [May 13, 2025, 7:16pm UTC](https://forums.percona.com/t/pmm-3-1-0-alertmanager-tls-certificate-trust-issue-with-proxy-slack-webhook/37906/2 "2025-05-13T19:16:33Z")
Hello, we dropped support of internal alertmanager in PMM 3, currently we support external alertmanagers only through [Grafana UI](https://grafana.com/docs/grafana/latest/alerting/set-up/configure-alertmanager/). > x509: certificate signed by unknown authority Can you share where do you see this error? ---
**Author:** ![pmm3.1](https://avatars.discourse-cdn.com/v4/letter/p/8edcca/32.png) [@pmm3.1](https://forums.percona.com/u/pmm3.1)\ **Post date:** [May 15, 2025, 2:25am UTC](https://forums.percona.com/t/pmm-3-1-0-alertmanager-tls-certificate-trust-issue-with-proxy-slack-webhook/37906/3 "2025-05-15T02:25:48Z")
Hello, This proxy uses an internally signed certificate, which results in the following TLS error when sending test alerts ![스크린샷 2025-05-15 오전 11.16.55(2)](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/3X/b/f/bf19486f207374c260ac164091c47f5b2883ad81.jpeg) What I’ve learned so far: - I understand that PMM 3.x has deprecated internal Alertmanager, and that all alerting is now handled via Grafana Alerting (Unified Alerting). - The error above seems to originate from Grafana’s Slack notification contact point, which attempts to send a message through the proxy but fails to validate the proxy’s TLS certificate. What would be the recommended way to allow Slack notifications to pass through the proxy without TLS failures? While investigating further, I noticed that **`vmalert` is running as part of the PMM 3.1.0 container** , alongside Grafana and other components. ### Does `vmalert` handle any of the alert notification delivery (e.g., Slack)? - Or is all outbound alerting handled exclusively by **Grafana Alerting / Contact Points** in PMM 3.x? Additionally, if `vmalert` is involved in sending alerts externally: - Is it possible to configure `vmalert` to **use a corporate proxy** for outbound requests (e.g., via environment variables like `HTTP_PROXY`, or via CLI flags)? - Can `vmalert` be configured to **trust a custom CA certificate** for TLS communication (similar to how system CA bundles are used)? Thanks again for your continued support! ---
**Author:** ![nurlan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/nurlan/32/1523_2.png) [@nurlan](https://forums.percona.com/u/nurlan)\ **Post date:** [May 15, 2025, 9:24pm UTC](https://forums.percona.com/t/pmm-3-1-0-alertmanager-tls-certificate-trust-issue-with-proxy-slack-webhook/37906/4 "2025-05-15T21:24:41Z")
Hi, we left vmalert only for recording rules. We had plans to use recording rules for better performance. it doesn’t communicate with Grafana unified alerting. I’ve found Github issue which might be related to [your one](https://github.com/grafana/grafana/issues/82779)