netdata / netdata

The open-source observability platform everyone needs!

Home Page:https://www.netdata.cloud

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feat]: Who watches the watchers?

wmertens opened this issue · comments

Problem

I have apps that expose endpoints and I need to be sure the apps are running.

Description

I want to be able to mark alert as required, together with an instance. If there's no matching alert for that instance, there should be an alert saying it's missing.

Importance

really want

Value proposition

when you have a bunch of services that are supposed to be working, but netdata doesn't know about them until they are detected, you won't know until you check manually.

Proposed implementation

Perhaps one of:

  • add the property required chart labels to alert definitions: an array of conditions that should match at least one alert. Then you can define one entry per known instance
  • a separate configuration that lists instance+alert label pairs that have to exist at all times