Graphite reporter

[Alvaro Rodriguez]

It would be nice to have a Graphite reporter / adapter like the Java metrics library does.
I know it's on your TODO in the home page, I'm just upgrading it to issue status.

[Iulian Margarintescu]

Yes, that is on the TODO list, but since setting up Graphite on windows is a major pain, it might get delayed a bit. From what I've seen in the java version of the metrics library the code for pushing data to Graphite is not complicated, but you still need Graphite.

In one of my tries, i've found a vangard image with a pre-configured Graphite, but there were all sorts of errors in accessing it and I gave up then.

I will get back to it in the future, but I can't promise anything :)

[Alvaro Rodriguez]

I'm looking at bringing the original Metrics Graphite reporting to this library.
Things are a bit different there. Could we discuss here how best to port the code?
For example Reporters use a MetricFilter to decide which metrics to report.
MetricFilter has this signature:

bool Matches(String name, Metric metric);

where Metric is a base interface for all metrics. There is no such thing here. Do you think that the metrics could implement a base interface like this, or maybe you have a different concept here that we could use. I'm just getting into the internals of the libraries so I'd appreciate your input.

[Iulian Margarintescu]

So far I had mixed thoughts about the concept of MetricFilter from the original port.

I'm not sure filtering metrics should be the responsibility of metrics "capturing" app. It might make sense to only send some metrics to a specific receiver, but i'm not convinced yet that it is worth the complexity.

I also had mixed thoughts about a common Metric interface and finally decided to not have it. It only makes sense for metric consumers and not for metric producers - and one of the goals of the lib was to keep the metric collecting api as clean and as minimal as possible, since this is the part that gets used in all the parts of the host application where metrics need to be collected.

The metric reporters on the other hand are defined in the library (and can afford to deal with more complexity) and are usually configured in one place in the host library.

For the Graphite adapter - I would suggest reporting all the metrics at first and then, if metric categories or metric "sets" are implemented or maybe even filters, they can be added later.

Looking at the Java implementation, the .net port should not be very different. Not sure if or how Graphite handles Units or if it even cares about them.

[Jules Van de Velde]

I had the same need to be able to send metrics to Graphite and decided to take a stab at creating a reporter. Right now, it depends on an external client (Graphite-net) for communication with Graphite, although porting the original java code shouldn't be too difficult. You can check it out at jvandevelde/Metrics.NET@8ce0695

I also provided a way to prefix all metrics sent by the application with a 'metricPrefix' argument to the GraphiteReporter

@etishor AFAIK, Graphite does not handle units so it turns into a matter of simply sending a metric name and value to the server

[Iulian Margarintescu]

@jvandevelde that looks interesting.

This weekend I've also started to work on a few ways of sending metrics to graphite (running in a linux vm) and statsd. I'll share my results soon.

Sending metrics directly to graphite, using UDP or TCP, without any dependency looks to be straight forward. StatsD is a bit different as it does its own metrics aggregation and it would make sense to have a way of sending it things like actual counter increments or actual captured duration values.

This got me to think about the larger picture. There are basically to approaches for the problem:

1. you collect & aggregate metrics in process and on a timer push the values to some other service (or some other service retrieves the metrics using json over http - I've also have this working as a POC)
2. you buffer the actual operations ( increments, duration etc ) and push them to another service which does the aggregation & persistence. This means the in process metrics are just an interface for collecting and dispatching metrics to some other service.

At the moment I really want to think about these scenarios a bit more and come up with a plan for handling them.

[Jules Van de Velde]

Agreed, sending metrics directly to Graphite without an external dependency shouldn't be very difficult. My inclusion of Graphite-net was simply to get a working example going quickly.

I'm interested to see what you have in mind for integration between StatsD and the Metrics.NET library. It seems to me that Metrics.NET does a lot of the same things that StatsD does (aggregate counters, timers, calculate histograms) over a configured interval. I can see several scenarios where using both could be beneficial though:

1. If you wanted to sample and report on metrics for different intervals in Metrics.NET reporters and Graphite. Having StatsD in between the two allows you to aggregate in a way that you won't lose data when sending the individual stats to Carbon, which can only record 1 value per configured interval.
2. If you wanted to allow multiple processes collect and aggregate data for the same metric, though this could also be handled with namespaces and wildcard support in Graphite's charting support.

[Iulian Margarintescu]

Not quite complete, but working: https://github.com/etishor/Metrics.NET/tree/feature/graphite/Src/Metrics/Graphite

I still have some un-pushed changes to that branch, but it should be working.

Hardest part by far was getting graphite to run.

I'll hope to finish the initial implementation this weekend and push a new version.

[Iulian Margarintescu]

The Graphite bits are in https://github.com/etishor/Metrics.NET/tree/master/Src/Metrics/Graphite

Metric.Config
    .WithReporting(r => 
        r.WithGraphite(new Uri("net.udp://graphite.myhost.com:2003"), TimeSpan.FromSeconds(1)));

For now there is support for UDP & TCP.

I only tested this with graphite running inside a VM and there might still be issues with the current implementation.