ztxz16 / fastllm

纯c++的全平台llm加速库,支持python调用,chatglm-6B级模型单卡可达10000+token / s,支持glm, llama, moss基座,手机端流畅运行

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

qwen1.5 int4模型回复出现解码问题:UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 72-73: invalid continuation byte

zhang415 opened this issue · comments

出错代码为:(最后一行报错)
for i, his in enumerate(history):
round_idx = 0
for (q,a) in his:
history_str[i] = g_model.make_history(history_str[i], round_idx, q, a)
round_idx += 1
final_round = round_idx
prompts = [g_model.make_input(history_str[i], final_round, prompts[i])[-args.max_input_length:] for i in range(len(prompts))]
results = []
for response in g_model.batch_response(prompts, None, config):

报错:
6

7
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 72-73: invalid continuation byte

输入格式为:{
"contents":["反光板复活复活复活南方女孩妇女和功能和发v飞鸟飞过发表过打不过打不过打不过打不过打不过打不过打不过打不过打不过打不过的不敢当不敢当不敢当不敢当不敢当不敢当吧"],
"prompt":[[["今天星期几","三"],["今天星期几", "四"]]],
"prefix":["给出结果"],
"system":"现在你是一名秘书"
}