simonw / llm-mistral

To figure out the new model IDs:

export MISTRAL=$(cat "$(llm keys path)" | jq .mistral -r)
curl -s 'https://api.mistral.ai/v1/models' -H "Authorization: Bearer $MISTRAL" | jq '.data[].id'

Outputs:

"open-mistral-7b"
"mistral-tiny-2312"
"mistral-tiny"
"open-mixtral-8x7b"
"mistral-small-2312"
"mistral-small"
"mistral-small-2402"
"mistral-small-latest"
"mistral-medium-latest"
"mistral-medium-2312"
"mistral-medium"
"mistral-large-latest"
"mistral-large-2402"
"mistral-embed"

Current code:

llm-mistral/llm_mistral.py

Lines 8 to 12 in a4f6400

    
           @llm.hookimpl 
        
           def register_models(register): 
        
               register(Mistral("mistral-tiny")) 
        
               register(Mistral("mistral-small")) 
        
               register(Mistral("mistral-medium"))

A bit annoying that they don't have a mistral-large alias - the closest is mistral-large-latest.

I'm inclined to keep my existing mistral-tiny and mistral-small and mistral-medium aliases, add my own mistral-large that points to their latest, then make ALL of their model IDs available as things like mistral/mistral-medium-2312.

I'll fetch their JSON file the first time you try to call one of their models and cache it in mistral-models.json, then provide a llm mistral refresh command that refreshes it.

Extract from the end of that JSON:

  [
    {
      "id": "mistral-large-2402",
      "object": "model",
      "created": 1708960437,
      "owned_by": "mistralai",
      "root": null,
      "parent": null,
      "permission": [
        {
          "id": "modelperm-d7f053a61bc14064b9b3a66c5e2735ed",
          "object": "model_permission",
          "created": 1708960437,
          "allow_create_engine": false,
          "allow_sampling": true,
          "allow_logprobs": false,
          "allow_search_indices": false,
          "allow_view": true,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ]
    },
    {
      "id": "mistral-embed",
      "object": "model",
      "created": 1708960437,
      "owned_by": "mistralai",
      "root": null,
      "parent": null,
      "permission": [
        {
          "id": "modelperm-ccf99400057448afb0567916e545c8a0",
          "object": "model_permission",
          "created": 1708960437,
          "allow_create_engine": false,
          "allow_sampling": true,
          "allow_logprobs": false,
          "allow_search_indices": false,
          "allow_view": true,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ]
    }
  ]

The mistral-embed model doesn't seem to have any metadata that help distinguish it from the non-embedding models, so I'll have to special case that.

	@llm.hookimpl
	def register_models(register):
	register(Mistral("mistral-tiny"))
	register(Mistral("mistral-small"))
	register(Mistral("mistral-medium"))

Support the new Mistral Large model