grosser / kennel

Datadog monitors/dashboards/slos as code, avoid chaotic management via UI

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Thoughts

WillPlatnick opened this issue · comments

Hello,
We're interested in using kennel as the basis of our Monitoring as Code project. Our use case is slightly different than how kennel is written.

We'd like to be able to use kennel as the starting point to create our dashboards, but we don't want our developers to be limited to updating them in code. We'd like developers to be able to update the dashboards themselves within the Datadog UI, and then use kennel to grab the changed dashboard and commit the changes into version control.

I believe this would require some changes to kennel to do well. For instance, we want the ability to add a dashboard using kennel, but not update ones that haven't been pulled into version control yet. We'd also like to be able to import existing stuff by kennel_id to find the datadog id.

As I go down the path of building this out, I wanted to know if the changes I make would be welcome to kennel or if you would like to keep this functionality out because it violates what you're trying to build.

The usual workflow I see is "create in the UI" + import, since it's hard to get the queries etc right without the UI.

Making changes in the UI and then automatically pulling/committing prevents use of comments/nice formatting/reuse, so I'm not a fan of that workflow, also it often ends up being forgotten and then there is a "sync" commit that updates everything without background.

"not update ones that haven't been pulled into version control yet", that does not happen since kennel only updates things it created.

"import by kennel id" sounds good/simple, so PR welcome, and is also a nice workaround to find the id of something.

The workflow I'm thinking is that terraform kicks off a script that creates the kennel files and then uses kennel to apply them. If a user had updated a dashboard manually that was previously created by kennel but hadn't been imported, we'd like not overwrite their changes.

Maybe better to use the terraform datadog integration ?
Creating but not importing things is not really a workflow kennel supports 🤷‍♂