prometheus / jmx_exporter

A process for exposing JMX Beans via HTTP for Prometheus consumption

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Blacklisting Percentile Metrics

anderoo opened this issue · comments

Hello, is it possible to blacklist metrics for certain percentiles or attributes? I'd like to drop 75thPercentile and 95thPercentile metrics for Cassandra. For example:org.apache.cassandra.metrics:type=Table,keyspace=*,scope=*,name=RangeLatency Attribute=75thPercentile.

I've tried excludeObjectNameAttributes and Blacklist with various patterns to no success

@anderoo Currently, the mechanism to exclude specific attributes (excludeObjectNameAttributes) requires the full ObjectName. Wildcards are not supported.

Why are wildcards not supported?

Using wildcards would require a regex match for every ObjectName / attribute name combination (fully qualified attribute.)

Considering the number of possible metrics, this would drastically kill performance.

@dhoard Thanks for the reply. Is the wildcard logic not the same for blacklisting then? I currently have a few wildcard blacklists with the intention of improving performance, and my goal with dropping certain percentiles is to improve performance further and reduce the number of metrics scraped.

I'll see if I can get these working with full object names in the meantime, but given labels like "table" would be fairly flexible, it's probably not a solution.

@anderoo The MBean ObjectName doesn't contain the names of attributes of the MBean.

To filter by only ObjectName / only attribute name or the combination would require building a string for every ObjectName/attribute combination, and then performing the regex filtering which would be expensive.

I would double-check how Cassandra implements the MBean. If they periodically calculate the value and store it, then the call to get the value should be quick. If they are calculating it "inline", then the call to get the value is expensive.