Logstash reporter
jenyayel opened this issue · comments
Hi @alhardy,
We figured out that writing directly to ES index using #13 won't work best for our needs. Instead, we decided to write to Logstash and ES reporter is not that suitable for this.
I can propose 2 things that can take advantage of HTTP plugin:
- Add additional client to ES reporter and the way to configure which one to use
- Add separate generic HTTP reporter. This is similar how it is done in Serilog
There are obviously other options, I just thought those two will make more sense to the broad audience. LMK what you think and I'll gladly contribute.
@jenyayel Interested to know why going direct to ES doesn't work for you guys? Is it that you want a central place to index your metrics from multiple sources?
What are your ideas on a generic HTTP reporter? Would it take some type of serialization abraction?
We have a central place and a same way for managing logs and metrics (app and infra). Wide range of technologies and formats, so it makes sense for us to have a central middleware to process logs and metrics as they come before they are stored.
I didn't have any special ideas about HTTP reporter. I took a look at Serilog and they have something really straight forward with ability to inject HttpClient which should be fine. Regarding serialization, JSON probably would be fine for most of the cases.
BTW, do you have any plans on adding abstraction for formatting console reporter?
OK, HTTP reporter should be easy enough, see the changes I've made to InfluxDbReporterProvider. Clients could take a dependancy on different formatters based on how they want to report over HTTP e.g. App.Metrics.Formatters.Json, the Prometheus formatter will have ProtoBuf serialization.
There's no abstraction yet for the console reporter, but log an issue I'll and I'll have look as part of the next release.
Sounds like a plan. I'll try to get my hands on it soon.
P.S. Sorry for delayed replies - busy period for me now.
All good @jenyayel, I should be able to get to the HTTP reporter late next week, early the week after if you dont beat me too it. Keep the feature requests coming! ;)
The HTTP reporter should also be useful for those using Influx Telegraph as well.
BTW have you guys looked at metricbeats as a logstash alternative for this type of thing?
Thinking a generic HTTP reporter can live here, and serialization can live in a formatter project e.g. Json and Prometheus. ES/Logstash specific serialization can go in the Elasticsearch repo
@jenyayel sorry this took a while before I could get to it. The HTTP reporter is done, you can look at the CustomMetricPayloadBuilder on how to implement a custom payload.
I've moved all reporter specific formatting into their own packages e.g. App.Metrics.Formatting.InfluxDB and App.Metrics.Formatting.ElasticSearch, I guess there would be a App.Metrics.Formatting.Logstash package as well which could be used with the HTTP reporter.
See here for the HTTP Reporter feature.
Amazing! I'm going to try it this week. Thank you very much. As usual an outstanding work.
@jenyayel no time yet? (understand time is limited being an engineer). I'm really interested to see this action ;)
BTW, this is all quite flexible now, you could also use the text file reporter writing metrics to disk for logstash to pick up, or you could have the /metrics or /metrics-text return metrics ES formatted and take a pull based approach.
I'd also be keen at some stage getting something going with beats. Have you guys tried that as a logstash replacement?