H11 Got data when expected EOF

[Jeremy Lyman]

I am running Hypercorn from the cli with Trio and my application layer is FastAPI. During a security endpoint scan that is outside of my control, my application always fails with the following exception at the h11/hypercorn trio layer. I don't have much insight into the requests being made and nothing stands out from the access logs. I am not able to reproduce this issue with normal interactions with the application so I thought I'd reach out to see if anyone might have any thoughts on how to debug or has an understanding on what is occurring that might cause h11 / trio to get into this state.

hypercorn -c file:hypercorn_conf.py main:application

worker_class = "trio"
workers = 6
graceful_timeout = 3.0
read_timeout = 5
shutdown_timeout = 2.0

Process SpawnProcess-1:
Traceback (most recent call last):
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/protocol/h11.py", line 157, in _handle_events
    event = self.connection.next_event()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/h11/_connection.py", line 487, in next_event
    exc._reraise_as_remote_protocol_error()
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/h11/_util.py", line 77, in _reraise_as_remote_protocol_error
    raise self
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/h11/_connection.py", line 469, in next_event
    event = self._extract_next_receive_event()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/h11/_connection.py", line 411, in _extract_next_receive_event
    event = self._reader(self._receive_buffer)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/h11/_readers.py", line 223, in expect_nothing
    raise LocalProtocolError("Got data when expecting EOF")
h11._util.RemoteProtocolError: Got data when expecting EOF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/trio/run.py", line 127, in trio_worker
    trio.run(partial(worker_serve, app, config, sockets=sockets, shutdown_trigger=shutdown_trigger))
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/trio/_core/_run.py", line 2275, in run
    raise runner.main_task_outcome.error
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/trio/run.py", line 50, in worker_serve
    async with trio.open_nursery() as server_nursery:
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/trio/_core/_run.py", line 971, in __aexit__
    raise combined_error_from_nursery
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/trio/_highlevel_serve_listeners.py", line 33, in _run_handler
    await handler(stream)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/trio/tcp_server.py", line 70, in run
    await self._read_data()
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/trio/tcp_server.py", line 106, in _read_data
    await self.protocol.handle(RawData(data))
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/protocol/__init__.py", line 62, in handle
    return await self.protocol.handle(event)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/protocol/h11.py", line 113, in handle
    await self._handle_events()
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/protocol/h11.py", line 160, in _handle_events
    await self._send_error_response(error.error_status_hint)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/protocol/h11.py", line 251, in _send_error_response
    await self._send_h11_event(
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/protocol/h11.py", line 248, in _send_h11_event
    await self.send(RawData(data=data))
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/hypercorn/trio/tcp_server.py", line 82, in protocol_send
    await self.stream.send_all(event.data)
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/trio/_highlevel_socket.py", line 104, in send_all
    with self._send_conflict_detector:
  File "/opt/pysetup/.venv/lib/python3.11/site-packages/trio/_util.py", line 217, in __enter__
    raise trio.BusyResourceError(self._msg)
trio.BusyResourceError: another task is currently sending data on this SocketStream