add daemon osd.X perf dump metrics
mbrt opened this issue · comments
Metrics coming from ceph daemon osd.X perf dump are very important to evaluate performace. The only problem I see in the code is that for now there is no other case where you need to call an API for every OSD, as this case requires. Is there any plan to support them?
Those metrics are gathered from local daemons running on each node whereas ceph_exporter is designed to work only with the monitors at the moment.
I suppose an easier way to do what you want is to make a small json parser to call from cron and just dump the output to a text file to be served by a node_exporter instead using the textfile capability.
A similar issue is extracting metrics from the rados gateway which also uses a separate API.
Yeah, that is definitely possible. I still think that having a separate executable or a different execution mode for the ceph_exporter to make it run on every osd node would be useful, because by default it provides only basic information, so it's somehow incomplete. Is it something to be considered for the future, or out of scope?
Different execution mode is something we can surely add. Currently in our setup, we have a different binary that extracts stats from perf dump. I am also planning to keep tabs on prometheus module of ceph-mgr as this work might be moot if the ability to extract OSD-level perf stats can be added to ceph-mgr.
@neurodrone I'm also looking at ceph-mgr, but first we have to upgrade to Luminous 😄. Alright, I'll keep this issue open then, because it seems to make some sense.
Is there any result?
As far as I can tell, there's still no way to access these metrics from ceph-mgr. Closing this issue since it doesn't look like it's going to be available any time soon.
You can get these metrics on ceph masters by running ceph tell osd.\* perf dump, but not sure if that's a good solution.
@madushan1000, yes, it turns out that as of Ceph Octopus these are available via ceph tell. If this issue is still valuable to someone, they should feel free to re-open it for eventual investigation into feasibility for Octopus+ support.