marqo-ai / marqo

Unified embedding generation and search engine. Also available on cloud - cloud.marqo.ai

Home Page:https://www.marqo.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[ENHANCEMENT] More descriptive `get_loaded_models` endpoint

vicilliar opened this issue · comments

commented

Is your feature request related to a problem? Please describe.
Currently, get_loaded_models returns model_device and model_name, which is based on the alias given as model in index defaults by a user. This is good when the model is in the registry, as the model alias usually matches the model_properties["name"], however this isn't always the case. Especially when users use custom models, it may be worth it to return more information in get_loaded_models.

For example, if an index is defined with:

settings={
    "index_defaults": {
        "model": "my_custom_model",
        "model_properties": {
            "name": "ViT-B-32-quickgelu",
            "dimensions": 512,
            "url": "https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_avg-8a00ab3c.pt",
            "type": "open_clip"
        }
    }
}

Currently get_loaded_models will only give you:

[
    {
        "model_name": "my_custom_model",
        "model_device": "cpu"
    }
]

Describe the solution you'd like
We should return other valuable info about the model, including:

  1. memory size
  2. model_properties["name"], as it's usually more useful than just model
  3. URL downloaded from (if it's a custom model)