量化后的ChatGlm3-6B使用open_api部署tools功能无法使用

[LGBG]

• 现象：
  使用open_api部署后，测试tools功能，请求体如下

{
    "functions": {
        "random_number_generator": {
            "name": "random_number_generator",
            "description": "Generates a random number x, s.t. range[0] <= x < range[1]",
            "params": [
                {
                    "name": "seed",
                    "description": "The random seed used by the generator",
                    "type": "int",
                    "required": true
                },
                {
                    "name": "range",
                    "description": "The range of the generated numbers",
                    "type": "tuple[int, int]",
                    "required": true
                }
            ]
        },
        "get_weather": {
            "name": "get_weather",
            "description": "Get the current weather for `city_name`",
            "params": [
                {
                    "name": "city_name",
                    "description": "The name of the city to be queried",
                    "type": "str",
                    "required": true
                }
            ]
        }
    },
    "model": "chatglm3-6b",
    "messages": [
        {
            "role": "user",
            "content": "帮我查询北京的天气如何"
        }
    ]
}

预期：

• 希望返回function的相关信息，例如

"function_call": {
                    "name": "get_weather",
                    "arguments": "{\"city_name\": \"北京\"}"
                }

实际：

• 实际返回如下

{
    "id": "chatcmpl",
    "model": "default-model",
    "object": "chat.completion",
    "created": 1703834923,
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "很抱歉，作为一个人工智能助手，我无法实时查询天气信息。您可以使用手机应用或搜索引擎查询最新的北京天气。"
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 12,
        "completion_tokens": 30,
        "total_tokens": 42
    }
}

补充：使用同样的请求参数访问非量化chatglm3-6B，可以正确返回相关信息

{
    "model": "chatglm3-6b",
    "object": "chat.completion",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "get_weather\n ```python\ntool_call(city_name='北京')\n```",
                "name": null,
                "function_call": {
                    "name": "get_weather",
                    "arguments": "{\"city_name\": \"北京\"}"
                }
            },
            "finish_reason": "function_call"
        }
    ],
    "created": 1703835057,
    "usage": {
        "prompt_tokens": 285,
        "total_tokens": 307,
        "completion_tokens": 22
    }
}

[LGBG]

但是使用cli运行看起来又可以正常使用

Welcome to ChatGLM.cpp! Ask whatever you want. Type 'clear' to clear context. Type 'stop' to exit.

System   > Answer the following questions as best as you can. You have access to the following tools:
{
    "random_number_generator": {
        "name": "random_number_generator",
        "description": "Generates a random number x, s.t. range[0] <= x < range[1]",
        "params": [
            {
                "name": "seed",
                "description": "The random seed used by the generator",
                "type": "int",
                "required": true
            },
            {
                "name": "range",
                "description": "The range of the generated numbers",
                "type": "tuple[int, int]",
                "required": true
            }
        ]
    },
    "get_weather": {
        "name": "get_weather",
        "description": "Get the current weather for `city_name`",
        "params": [
            {
                "name": "city_name",
                "description": "The name of the city to be queried",
                "type": "str",
                "required": true
            }
        ]
    }
}
Prompt   > 帮我查询北京的天气如何
ChatGLM3 > get_weather
```python
tool_call(city_name='北京')

Function Call > Please manually call function get_weather and provide the results below.
Observation > 温度 5度，天气晴，湿度 50
ChatGLM3 >
北京的天气查询结果如下：当前气温为5度，天气晴朗，湿度为50%。

[Longhao.Chen]

我也遇到了这个问题，请问有什么新的进展吗？