知识库问答报错Tool output: Range of input length should be [1, 6000]

[Jason Miller]

使用阿里在线模型，涉及到知识库问答的时候会报错Tool output: Range of input length should be [1, 6000]

Tool input: {'query': "我是大学教授，年收入40万，无其他收入，需要交多少税”}
Tool output: Range of input length should be [1, 6000]
通过streaming=ture开启了千问的流式输出，顺便说一下文档里面llm_request:stream: true开启流式输出对千问在线模型无效

[2024-05-29 10:57:30.698556]|ERROR|BISHENG|None|8|139641528252096|'NoneType' object has no attribute 'get'
Traceback (most recent call last):

File "/usr/local/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
│ └ <function Thread._bootstrap_inner at 0x7f017fdb8e50>
└ <Thread(Thread-231 (start_loop), started 139641528252096)>
File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
│ └ <function Thread.run at 0x7f017fdb8b80>
└ <Thread(Thread-231 (start_loop), started 139641528252096)>
File "/usr/local/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
│ │ │ │ │ └ {}
│ │ │ │ └ <Thread(Thread-231 (start_loop), started 139641528252096)>
│ │ │ └ (<uvloop.Loop running=True closed=False debug=False>,)
│ │ └ <Thread(Thread-231 (start_loop), started 139641528252096)>
│ └ <bound method ThreadPoolManager.start_loop of <bisheng.utils.threadpool.ThreadPoolManager object at 0x7f013ebf5bd0>>
└ <Thread(Thread-231 (start_loop), started 139641528252096)>

File "/app/bisheng/utils/threadpool.py", line 72, in start_loop
loop.run_forever()
│ └ <method 'run_forever' of 'uvloop.loop.Loop' objects>
└ <uvloop.Loop running=True closed=False debug=False>

File "/app/bisheng/chat/handlers.py", line 37, in dispatch_task
await self.handler_dict[action](session, client_id, chat_id, payload, user_id)
│ │ │ │ │ │ │ └ 1
│ │ │ │ │ │ └ {'nodes': [{'id': 'MixEsVectorRetriever-J35CZ', 'data': {'id': 'MixEsVectorRetriever-J35CZ', 'node': {'template': {'tags': {'...
│ │ │ │ │ └ None
│ │ │ │ └ 'ae106b38-61bb-4108-96dc-07e0e5d0b798'
│ │ │ └ <bisheng.chat.manager.ChatManager object at 0x7f013e90ef80>
│ │ └ 'default'
│ └ {'default': <bound method Handler.process_message of <bisheng.chat.handlers.Handler object at 0x7f00b69517e0>>, 'autogen': <b...
└ <bisheng.chat.handlers.Handler object at 0x7f00b69517e0>

File "/app/bisheng/chat/handlers.py", line 137, in process_message
result, intermediate_steps, source_doucment = await process_graph(
└ <function process_graph at 0x7f013ebf31c0>

File "/app/bisheng/chat/utils.py", line 41, in process_graph
result, intermediate_steps, source_document = await get_result_and_steps(
└ <function get_result_and_steps at 0x7f013e9840d0>

File "/app/bisheng/processing/base.py", line 29, in get_result_and_steps
output = await langchain_object.acall(inputs, callbacks=async_callbacks)
│ │ │ └ [<bisheng.api.v1.callback.AsyncStreamingLLMCallbackHandler object at 0x7f00b744d4e0>]
│ │ └ {'question': '我是大学教授，年收入40万，无其他收入，需要交多少税'}
│ └ <function Chain.acall at 0x7f0164ac5090>
└ ConversationalRetrievalChain(memory=ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content='国...

File "/usr/local/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper
return await wrapped(*args, **kwargs)
│ │ └ {'callbacks': [<bisheng.api.v1.callback.AsyncStreamingLLMCallbackHandler object at 0x7f00b744d4e0>]}
│ └ (ConversationalRetrievalChain(memory=ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content='...
└ <function Chain.acall at 0x7f0164ac4b80>
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall
return await self.ainvoke(
│ └ <function Chain.ainvoke at 0x7f0164ac4310>
└ ConversationalRetrievalChain(memory=ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content='国...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke
raise e
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke
await self._acall(inputs, run_manager=run_manager)
│ │ │ └ <langchain_core.callbacks.manager.AsyncCallbackManagerForChainRun object at 0x7f00b6a83eb0>
│ │ └ {'question': '我是大学教授，年收入40万，无其他收入，需要交多少税', 'chat_history': 'Human: 国家税务总局2024年网站日常内容维护服务项目的编号\nAI: 项目编号为：ZB2024-ZXJC-C0005-B0...
│ └ <function ConversationalRetrievalChain._acall at 0x7f01655c00d0>
└ ConversationalRetrievalChain(memory=ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content='国...
File "/usr/local/lib/python3.10/site-packages/bisheng_langchain/chains/conversational_retrieval/base.py", line 106, in _acall
answer = await self.combine_docs_chain.arun(input_documents=docs,
│ │ │ └ [Document(page_content='13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E6）其他扣除：□□,□□□,□□□,□□□.□□（元） □无此类扣除...
│ │ └ <function Chain.arun at 0x7f0164ac5750>
│ └ StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{conte...
└ ConversationalRetrievalChain(memory=ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content='国...
File "/usr/local/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper
return await wrapped(*args, **kwargs)
│ │ └ {'input_documents': [Document(page_content='13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E6）其他扣除：□□,□□□,□...
│ └ (StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{cont...
└ <function Chain.arun at 0x7f0164ac4dc0>
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 627, in arun
await self.acall(
│ └ <function Chain.acall at 0x7f0164ac5090>
└ StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{conte...
File "/usr/local/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper
return await wrapped(*args, **kwargs)
│ │ └ {'callbacks': <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x7f00b6a806a0>, 'tags': None, 'metadata': None}
│ └ (StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{cont...
└ <function Chain.acall at 0x7f0164ac4b80>
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall
return await self.ainvoke(
│ └ <function Chain.ainvoke at 0x7f0164ac4310>
└ StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{conte...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke
raise e
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke
await self._acall(inputs, run_manager=run_manager)
│ │ │ └ <langchain_core.callbacks.manager.AsyncCallbackManagerForChainRun object at 0x7f00b747f790>
│ │ └ {'input_documents': [Document(page_content='13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E6）其他扣除：□□,□□□,□...
│ └ <function BaseCombineDocumentsChain._acall at 0x7f0164349e10>
└ StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{conte...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 153, in _acall
output, extra_return_dict = await self.acombine_docs(
│ └ <function StuffDocumentsChain.acombine_docs at 0x7f016434b010>
└ StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{conte...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 262, in acombine_docs
return await self.llm_chain.apredict(callbacks=callbacks, **inputs), {}
│ │ │ │ └ {'question': '作为大学教授，年收入40万人民币，我需要缴纳多少个人所得税？', 'context': '13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E...
│ │ │ └ <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x7f00b695b700>
│ │ └ <function LLMChain.apredict at 0x7f0164ac7be0>
│ └ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
└ StuffDocumentsChain(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{conte...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/llm.py", line 310, in apredict
return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]
│ │ │ │ │ └ 'text'
│ │ │ │ └ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
│ │ │ └ <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x7f00b695b700>
│ │ └ {'question': '作为大学教授，年收入40万人民币，我需要缴纳多少个人所得税？', 'context': '13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E...
│ └ <function Chain.acall at 0x7f0164ac5090>
└ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
File "/usr/local/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper
return await wrapped(*args, **kwargs)
│ │ └ {'callbacks': <langchain_core.callbacks.manager.AsyncCallbackManager object at 0x7f00b695b700>}
│ └ (LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
└ <function Chain.acall at 0x7f0164ac4b80>
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall
return await self.ainvoke(
│ └ <function Chain.ainvoke at 0x7f0164ac4310>
└ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke
raise e
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke
await self._acall(inputs, run_manager=run_manager)
│ │ │ └ <langchain_core.callbacks.manager.AsyncCallbackManagerForChainRun object at 0x7f00b69e0400>
│ │ └ {'question': '作为大学教授，年收入40万人民币，我需要缴纳多少个人所得税？', 'context': '13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E...
│ └ <function LLMChain._acall at 0x7f0164ac7ac0>
└ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/llm.py", line 275, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
│ │ │ └ <langchain_core.callbacks.manager.AsyncCallbackManagerForChainRun object at 0x7f00b69e0400>
│ │ └ {'question': '作为大学教授，年收入40万人民币，我需要缴纳多少个人所得税？', 'context': '13（E5）个人养老金：□□,□□□.□□（元） □无此类扣除\n29.您在纳税年度内发生的除上述扣除以外的其他扣除是多少？\n（E...
│ └ <function LLMChain.agenerate at 0x7f0164ac76d0>
└ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
File "/usr/local/lib/python3.10/site-packages/langchain/chains/llm.py", line 142, in agenerate
return await self.llm.agenerate_prompt(
│ │ └ <function BaseChatModel.agenerate_prompt at 0x7f01658ddc60>
│ └ ChatQWen(cache=True, client=Requests(headers={'Authorization': 'Bearer sk-ba503649c30d4a40bb7e65a94a58df6e', 'Content-Type': ...
└ LLMChain(prompt=PromptTemplate(input_variables=['context', 'question'], template='参考文本：\n{context}\n\n----------------------...
File "/usr/local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 570, in agenerate_prompt
return await self.agenerate(
│ └ <function BaseChatModel.agenerate at 0x7f01658ddb40>
└ ChatQWen(cache=True, client=Requests(headers={'Authorization': 'Bearer sk-ba503649c30d4a40bb7e65a94a58df6e', 'Content-Type': ...
File "/usr/local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 530, in agenerate
raise exceptions[0]
└ [AttributeError("'NoneType' object has no attribute 'get'")]
File "/usr/local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 715, in _agenerate_with_cache
result = await self._agenerate(
│ └ <function ChatQWen._agenerate at 0x7f01612136d0>
└ ChatQWen(cache=True, client=Requests(headers={'Authorization': 'Bearer sk-ba503649c30d4a40bb7e65a94a58df6e', 'Content-Type': ...
File "/usr/local/lib/python3.10/site-packages/bisheng_langchain/chat_models/qwen.py", line 315, in _agenerate
choices = output.get('choices')
└ None

AttributeError: 'NoneType' object has no attribute 'get'

[姚劲]

看着是给到模型的上下文长度太长，导致超过模型限制了。 可以通过combine_documents组件限制max-token方式进行约束