neuml / txtchat

💭 Retrieval augmented generation (RAG) and language model powered search applications

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[ERROR] default_exception_handler: Task exception was never retrieved

dominikstein-ibm opened this issue · comments

Hi All,

I got txtchat running by pip installing from source! However, more often than not I get below error. 1/10 times it works, 9/10 times below error is thrown.
My query is: "give me a quick summary on mars"
Persona: python -m txtchat.agent wikitalk.yml

just installed using:
pip install -U git+https://github.com/neuml/txtai
pip install -U git+https://github.com/neuml/txtchat

Any idea?

2023-03-03 09:42:53,976 [ERROR] default_exception_handler: Task exception was never retrieved
future: <Task finished name='Task-9' coro=<Dispatcher.call_method() done, defined at /Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py:22> exception=ConnectionClosedOK(Close(code=1000, reason='Normal closure'), Close(code=1000, reason='Normal closure'), True)>
Traceback (most recent call last):
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 28, in call_method
    await self._websocket.send(json.dumps(msg))
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1000 (OK) Normal closure; then sent 1000 (OK) Normal closure
2023-03-03 09:42:53,978 [ERROR] default_exception_handler: Task exception was never retrieved
future: <Task finished name='Task-8' coro=<RocketChat.send_message() done, defined at /Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py:54> exception=ConnectionClosedOK(Close(code=1000, reason='Normal closure'), Close(code=1000, reason='Normal closure'), True)>
Traceback (most recent call last):
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 56, in send_message
    await SendMessage.call(self._dispatcher, text, channel_id, thread_id)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/methods.py", line 113, in call
    await dispatcher.call_method(msg, msg_id)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 30, in call_method
    return await fut
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 46, in _process_incoming
    raise err
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 41, in _process_incoming
    await self._process_incoming_event()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 49, in _process_incoming_event
    msg = await self._websocket.recv()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 568, in recv
    await self.ensure_open()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1000 (OK) Normal closure; then sent 1000 (OK) Normal closure
2023-03-03 09:42:53,980 [ERROR] default_exception_handler: unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-2' coro=<RocketChat._start() done, defined at /Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py:31> exception=InvalidStateError('invalid state')>
Traceback (most recent call last):
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 38, in _start
    await self.dispatch_task
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 56, in send_message
    await SendMessage.call(self._dispatcher, text, channel_id, thread_id)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/methods.py", line 113, in call
    await dispatcher.call_method(msg, msg_id)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 30, in call_method
    return await fut
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 46, in _process_incoming
    raise err
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 41, in _process_incoming
    await self._process_incoming_event()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 49, in _process_incoming_event
    msg = await self._websocket.recv()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 568, in recv
    await self.ensure_open()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1000 (OK) Normal closure; then sent 1000 (OK) Normal closure

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 40, in _start
    connected_fut.set_exception(e)
asyncio.exceptions.InvalidStateError: invalid state
Traceback (most recent call last):
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/txtchat/agent/__main__.py", line 24, in <module>
    agent()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/txtchat/agent/base.py", line 42, in __call__
    self.start()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/txtchat/agent/rocketchat.py", line 47, in start
    asyncio.run(self.run())
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/txtchat/agent/rocketchat.py", line 69, in run
    await self.chat.run_forever()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 29, in run_forever
    await self.dispatch_task
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 38, in _start
    await self.dispatch_task
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/core.py", line 56, in send_message
    await SendMessage.call(self._dispatcher, text, channel_id, thread_id)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/methods.py", line 113, in call
    await dispatcher.call_method(msg, msg_id)
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 30, in call_method
    return await fut
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 46, in _process_incoming
    raise err
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 41, in _process_incoming
    await self._process_incoming_event()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/rocketchat_async/dispatcher.py", line 49, in _process_incoming_event
    msg = await self._websocket.recv()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 568, in recv
    await self.ensure_open()
  File "/Users/XXX/.pyenv/versions/3.8.13/lib/python3.8/site-packages/websockets/legacy/protocol.py", line 953, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1000 (OK) Normal closure; then sent 1000 (OK) Normal closure

I should add that the python agent crashes. In RocketChat there is no answer from the agent.

Can you retry with the latest source version? I'm undergoing a lot of changes between this and txtai. The hope is that things will start stabilizing to allow the next release.

At the bare minimum, the agent shouldn't crash this time and print a better error on what's happening.

just retried after fetching the latest source version. unfortunately still getting the same error

Ok, I will keep a look out. I have yet to see any connection closed errors like this. It's almost like it's hitting a socket timeout of some sort and the rc server is closing the connection.

With the custom process you're running, do responses take a while or are they quick to run?

yes, I was also thinking about some sort of time out. it usually takes some seconds until it crashes. However, I wasnt able to find a parameter that I can update to increase timeout limits. Maybe you have an idea? Would be happy to try it out.

Is this a Windows or macOS host? I've looked around a bit and it definitely looks like the connection is closed for some reason.

Its a macOS host. Thanks for checking! Do you have an idea which component does close the connection and what might be the reason?

Thanks for confirming. I don't really have any insight into why that message would occur. I'll link the rocketchat_async library author to see if he can provide any insights.

Reported this upstream with the rocketchat-async library.