通义千问流式输出的情况下function call异常
hejuntt1014 opened this issue · comments
晴天娃娃 commented
例行检查
- 我已确认目前没有类似 issue
- 我已确认我已升级到最新版本
- 我已完整查看过项目 README,尤其是常见问题部分
- 我理解并愿意跟进此 issue,协助测试和提供反馈
- 我理解并认可上述内容,并理解项目维护者精力有限,不遵循规则的 issue 可能会被无视或直接关闭
问题描述
通义千问流式输出的情况下function call异常,阿里官方文档显示incremental_output和stream是不支持同时设置的,但是源码中IncrementalOutput直接用request.Stream赋值了,这会使得Stream下function call输出异常
复现步骤
curl http://127.0.0.1:3000/v1/chat/completions \
-H "Authorization: Bearer sk-" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen1.5-72b-chat",
"stream": true,
"messages": [
{
"role": "user",
"content": "茂名天气怎样"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
}
}
],
"tool_choice": "auto"
}'
预期结果
希望在Stream流式输出的情况下获得openai官方api一致的输出结果
晴天娃娃 commented