windows_cpu_time_total decreasing occasionally?
jakeweston42 opened this issue · comments
We have windows-exporter feeding prometheus and then monitored with grafana and I'm kind of new to it all, but we've had some sporadic anomalies where the CPU reading will shoot way up briefly...above 100%, then drop back down to normal readings.
I whittled down the data to a specific core and mode, removed all the other calculations, and realized when I was doing a rate calculation on otherwise normal data, it skyrocketed at a very specific time.
So I looked a little closer at the underlying data and it appears the metric is decreasing a very small amount, and I'm wondering if Grafana (counting on it being a counter which never decreases) is having some math wonkiness trying to perform a rate() operation on a counter that seems to be decreasing.
This seems to have happened twice recently, both times they've been very tiny drops...one server one night did:
6939.3125
6939.3281
6939.3125
6939.3125
Another server, a different core (but also mode="priviledged")
4653.78125
4653.8125
4653.796875
4653.796875
4653.84375
My understanding was counters never go down. Any ideas why we seem to be getting a counter value decrease? Are there any easy fixes or work-arounds?
This isn't disruptive, but I'm trying to understand the data and anomalies better.
Thanks!
This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.