etsy / logster

Parse log files, generate metrics for Graphite and Ganglia

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

MetricLogster parser does not sum counts

fmahnke opened this issue · comments

When using MetricLogster parser, results for counts are different than I'd expect. Here's a simple example:

$ rm ~/logs/sample.log
$ echo " METRIC_COUNT metric=some.metric.count value=4 " >> ~/logs/sample.log
$ echo " METRIC_COUNT metric=some.metric.count value=4 " >> ~/logs/sample.log

$ ~/.virtualenvs/logster/bin/logster --log-dir=/home/asdf/logster --debug --state-dir=/home/asdf/logster --output=stdout MetricLogster /home/asdf/logs/sample.log
1427841578 some.metric.count 0.421052631579 # expect value to be 8

$ rm ~/logs/sample.log
$ echo " METRIC_COUNT metric=some.metric.count value=4 " >> ~/logs/sample.log
$ echo " METRIC_COUNT metric=some.metric.count value=6 " >> ~/logs/sample.log
$ echo " METRIC_COUNT metric=some.metric.count value=6 " >> ~/logs/sample.log

$ ~/.virtualenvs/logster/bin/logster --log-dir=/home/asdf/logster --debug --state-dir=/home/asdf/logster --output=stdout MetricLogster /home/asdf/logs/sample.log
1427841596 some.metric.count 0.75 # expect value to be 16

Is something going wrong with the call to get_state in MetricLogster? I don't totally understand that logic yet. Thanks!

I opened PR 85, which fixed this for me.

This behaviour is by design. As it's not possible to mandate an execution schedule for logster, we try to submit values per second where it makes sense to do so.

I'm looking through the supplied parsers and finding some that don't adhere to this approach, but it should be assumed that those are in error.

Please feel free to comment here if you'd like clarification or to discuss further