prometheus / alertmanager

Prometheus Alertmanager

Home Page:https://prometheus.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Templating support for body of webhook_config

fahlmant opened this issue · comments

It seems that many people want to have messaging platform-specific integrations, for example Issue #1365, and PR #1490. According to the documentation, this project is not accepting any new custom receivers. To remedy this, adding support for including a custom JSON body, similar to including a custom message template (as seen here) would give users the flexibility to add in whatever fields they need. This way, users don't have to run their own service to translate the body, and the maintainers don't have to maintain many custom receivers.

Similar to #1422

Also #701. This has been already discussed at length and the consensus so far has been to keep the webhook receiver as it is now. Although I see how it could be useful, it will also probably trigger additional requests such as custom authentication mechanisms, headers and so on. Eventually I'm afraid that it won't be that helpful.

Feel free to question this answer on our developer mailing list, but be aware it's unlikely that you will get a different answer.

@simonpasquier Is there a discussion on the mailing list, or just in issues and PRs?

@fahlmant I didn't find any discussion on the ML. @stuartnelson3 may know better.

To my knowledge, discussion has been only in issues/prs

This would be IMO very useful in case of e.g. MS Teams integration.

Currently you need to deploy yet another adapter service (https://github.com/idealista/prom2teams or https://github.com/prometheus-msteams/prometheus-msteams) whereas being able to define the webhook_config.text directly (e.g. using existing notification template functionality) would reduce this extra maintanence hasle.

Why would it be useful?

Consider a use case of monitoring local infrastructure's logs and getting notified on MS Teams.

To set up alerts on certain log events, you currently need to set up:

  • promtail (or redirect syslog + set up a central syslog server) on the end nodes
  • promtail to aggregate the data and let it be digested by Loki
  • Loki
  • Grafana
    • this is to visualize logs only, Loki datasource doesn't support defining alerts in Grafana, so you also need...
  • Alertmanager
  • an Alertmanager-to-MSTeams adapter service
  • MS Teams connector

These are a lot of moving parts just to monitor logs. With an ability of defining custom webhook template directly with Alertmanager, the adapter would be completely redundant.