Servercrash with autobahn testcase 7.1.6 on trio
apollo13 opened this issue · comments
The CI currently kills the server when running testcase 7.1.6:
[2023-12-29 12:12:57 +0100] [13470] [ERROR] Error in ASGI Framework
Traceback (most recent call last):
File "/home/florian/sources/hypercorn/src/hypercorn/trio/task_group.py", line 26, in _handle
await app(scope, receive, send, sync_spawn, call_soon)
File "/home/florian/sources/hypercorn/src/hypercorn/app_wrappers.py", line 34, in __call__
await self.app(scope, receive, send)
File "/home/florian/sources/hypercorn/server.py", line 9, in app
await send({
File "/home/florian/sources/hypercorn/src/hypercorn/protocol/ws_stream.py", line 283, in app_send
await self._send_wsproto_event(event)
File "/home/florian/sources/hypercorn/src/hypercorn/protocol/ws_stream.py", line 336, in _send_wsproto_event
data = self.connection.send(event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/florian/sources/hypercorn/.venv/lib/python3.12/site-packages/wsproto/connection.py", line 107, in send
raise LocalProtocolError(
wsproto.utilities.LocalProtocolError: Event TextMessage(data='Hello World!', frame_finished=True, message_finished=True) cannot be sent in state ConnectionState.CLOSED.
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib64/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib64/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/florian/sources/hypercorn/src/hypercorn/trio/run.py", line 122, in trio_worker
trio.run(partial(worker_serve, app, config, sockets=sockets, shutdown_trigger=shutdown_trigger))
File "/home/florian/sources/hypercorn/.venv/lib/python3.12/site-packages/trio/_core/_run.py", line 2093, in run
raise runner.main_task_outcome.error
File "/home/florian/sources/hypercorn/src/hypercorn/trio/run.py", line 46, in worker_serve
async with trio.open_nursery() as server_nursery:
File "/home/florian/sources/hypercorn/.venv/lib/python3.12/site-packages/trio/_core/_run.py", line 881, in __aexit__
raise combined_error_from_nursery
File "/home/florian/sources/hypercorn/.venv/lib/python3.12/site-packages/trio/_highlevel_serve_listeners.py", line 25, in _run_handler
await handler(stream)
File "/home/florian/sources/hypercorn/src/hypercorn/trio/tcp_server.py", line 55, in run
async with TaskGroup() as task_group:
File "/home/florian/sources/hypercorn/src/hypercorn/trio/task_group.py", line 76, in __aexit__
await self._nursery_manager.__aexit__(exc_type, exc_value, tb)
File "/home/florian/sources/hypercorn/.venv/lib/python3.12/site-packages/trio/_core/_run.py", line 881, in __aexit__
raise combined_error_from_nursery
File "/home/florian/sources/hypercorn/src/hypercorn/trio/task_group.py", line 39, in _handle
await send(None)
File "/home/florian/sources/hypercorn/src/hypercorn/protocol/ws_stream.py", line 260, in app_send
await self._send_wsproto_event(CloseConnection(code=CloseReason.INTERNAL_ERROR))
File "/home/florian/sources/hypercorn/src/hypercorn/protocol/ws_stream.py", line 336, in _send_wsproto_event
data = self.connection.send(event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/florian/sources/hypercorn/.venv/lib/python3.12/site-packages/wsproto/connection.py", line 107, in send
raise LocalProtocolError(
wsproto.utilities.LocalProtocolError: Event CloseConnection(code=<CloseReason.INTERNAL_ERROR: 1011>, reason=None) cannot be sent in state ConnectionState.CLOSED.
With asyncio the connection is closed and the server doesn't crash (though the "hello world" is not returned either):
Thinking more about this, there are probably two separate issues:
-
First one is the crash itself. I think hypercorn should never crash no matter whether the client is able to exploit a bug or not. Wouldn't it make sense to have a try/except at a very high level that allows to shut down a single connection in case of such errors.
-
The shutting down of the connection in combination with the asgi app still sending messages results in the mentioned traceback.
Fixed by 0bb4fb9
Also note workers are now restored (repopulated) if they crash.
This is lovely, thank you so much!