Giters

kserve / modelmesh

Distributed Model Serving Framework

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support individual model metrics

njhill opened this issue · comments

commented

Model-mesh currently exposes a comprehensive set of prometheus metrics, but those associated with a particular model are not currently labeled with the model id. This was an intentional decision since in the use cases model-mesh was designed for, there's a prohibitively large number of models managed, and many of them change frequently. Prometheus guidelines assert that the cardinality of label value permutations across all metrics should be constrained.

However, there are many usecases where the number of models managed is smaller, and for those it can be very useful to monitor metrics at the model level.

We should support this as a configurable option, either globally or per-model.

Some thoughts:

  • This would not apply to all published metrics. Some of them which would be of questionable use are TBD, such as req_queue_delay_milliseconds, age_at_eviction_milliseconds.
  • We could consider less granular labels such as the model type as an (additional) intermediate option. It might be more complicated however to determine the model type in the relevant metric publishing contexts. Also, the way that the type field is currently used within modelmesh-serving would limit the utility.
  • We need to think about what to do in relation to VModels - in many cases the VModel rather than Model would be of more interest. Probably configuration can take a form similar to labels=model,vmodel,type, so that an arbitrary combo of supported labels can be chosen.
commented

@ScrapCodes is looking into this

commented

@ScrapCodes: GitHub didn't allow me to assign the following users: VedantMahabaleshwarkar.

Note that only kserve members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @VedantMahabaleshwarkar

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.