JSON response format failing to retrieve clean JSON
serferdinand2 opened this issue · comments
Ognjen Ugljenović commented
Issues
When asking the model to return JSON via responseFormat (or response_format) setting in the request options, sometimes with JSON an additional text is added, usually explaining the JSON that it returned.
Reproduction:
String userPrompt
is generated dynamically based on the usecase.
const userPrompt = `Given the following Typescript Types: 'interface Question<T extends QuestionType> { title: string; type: T;
.....
, your job is to provide structured JSON formatted questions in an array. Create questions for a rating type question survey. Do not include any text in your response except JSON. Do not return Markdown or HTML. Be sure to create not more than 1 questions. Be sure to create at least 1 questions.`
mistral = new MistralClient(options.apiKey);
const chat = await mistral.chat({
model: 'mistral-small,
messages: [{ role: 'user', content: userPrompt }],
responseFormat: { type: 'json_object' },
temperature: temperature,
});
data = JSON.parse(
chat.choices[0].message.content
)
console.log(data)
Completion: {
id: '0ef53d7c6c3a4a5ebbaf43aa96c0f5d9',
object: 'chat.completion',
created: 1713366257,
model: 'mistral-small',
choices: [
{
index: 0,
message: {
role: 'assistant',
content: '[\n' +
'{\n' +
'"title": "How would you rate your overall satisfaction with our product?",\n' +
'"type": "rating",\n' +
'"required": true,\n' +
'"properties": {\n' +
'"labels": [\n' +
'{\n' +
'"label": "Very unsatisfied",\n' +
'"value": 1\n' +
'},\n' +
'{\n' +
'"label": "Unsatisfied",\n' +
'"value": 2\n' +
'},\n' +
'{\n' +
'"label": "Neutral",\n' +
'"value": 3\n' +
'},\n' +
'{\n' +
'"label": "Satisfied",\n' +
'"value": 4\n' +
'},\n' +
'{\n' +
'"label": "Very satisfied",\n' +
'"value": 5\n' +
'}\n' +
']\n' +
'}\n' +
'}\n' +
']\n' +
'\n' +
**'This JSON array contains one question of type "rating" that asks the customer about their overall satisfaction with the product. The question has 5 labels with corresponding values from 1 to 5, where 1 means "Very unsatisfied" and 5 means "Very satisfied". The question is required.',**
tool_calls: null
},
finish_reason: 'stop',
logprobs: null
}
],
usage: { prompt_tokens: 447, total_tokens: 658, completion_tokens: 211 }
}
Notes
The request header seems to be doing fine:
request {
model: 'mistral-small',
messages: [
{
role: 'user',
content: 'Given the following Typescript Types \
.......
Do not return Markdown or HTML. Be sure to create not more than 1 questions. Be sure to create at least 1 questions.'
}
],
tools: undefined,
temperature: 0,
max_tokens: undefined,
top_p: undefined,
random_seed: undefined,
stream: false,
safe_prompt: undefined,
tool_choice: undefined,
response_format: { type: 'json_object' }
}