mistralai / mistral-inference

Official inference library for Mistral models

Home Page:https://mistral.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

BUG: API /completion endpoint returns 500 (server error) when sending "max_token" = 1

MrXavier opened this issue · comments

While I was playing with the API endpoint /completion I found out a bug with the "max_tokens" body field when it's set to 1. Instead of returning 1 token response or a validation error, it returns 500 (internal server error).

If 1 token response is not allowed I believe it should return 400 bad request with the accepted boundaries for the max_token value.

Here is an example of request using curl:

curl --location "https://api.mistral.ai/v1/chat/completions" \
     --header 'Content-Type: application/json' \
     --header 'Accept: application/json' \
     --header "Authorization": Bearer {XXXY} \
     --data '{
    "model": "mistral-tiny",
    "messages": [
     {
        "role": "user",
        "content": “Prompt here…”
      }
    ],
    "max_tokens": 1
  }'