abersheeran / a2wsgi

Possibly related to #45 or the attempted fixes for it.

I have an ASGI stack that looks like (connexion framework):

Uvicorn -> Starlette (exception handling layer) -> request validation middleware -> WSGIMiddleware(FlaskApp) -> response validation middleware

Starting with version 1.10.1, if there is an exception in the response validation middleware layer, which is below the wrapped WSGI app, the app will hang and not return a response or be able to accept other incoming requests.

I can tell the exception never propagates back from the child ASGI middleware to the wrapped WSGI app, because if I add an exception handler the the WSGI Flask App, the exception never makes it there.

Can you provide a minimal reproducible example? Also, 1.10.2 has been released, please try the new version.

Here is I think something similar:

uvicornfail.py:

from a2wsgi import WSGIMiddleware
from flask import Flask
from starlette.applications import Starlette
from starlette.routing import Route

flask_app = Flask("test")


@flask_app.route("/healthz")
def healthz():
    return "OK"


app = Starlette(
    routes=[Route("/{path:path}", endpoint=WSGIMiddleware(flask_app.wsgi_app, workers=1))],
)

Using:

starlette==0.37.2
uvicorn==0.28.0 (0.27.1 seem to work though)
a2wsgi==1.10.2

Run with:
uvicorn uvicornfail:app --limit-concurrency 3

In another shell do something like:
while true; do curl localhost:8000/healthz ; done

Notice after a few requests, the uvicorn starts responding with "Service Unavailable". On the uvicorn logs there is nothing wrong yet. Now when you try to ctrl-c uvicorn, it starts outputting a series of tracebacks (presumably one for each hung request):

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/path/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/.venv/lib/python3.12/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/path/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/path/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/path/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/path/.venv/lib/python3.12/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/path/.venv/lib/python3.12/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/path/.venv/lib/python3.12/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/path/.venv/lib/python3.12/site-packages/a2wsgi/wsgi.py", line 166, in __call__
    return await responder(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/.venv/lib/python3.12/site-packages/a2wsgi/wsgi.py", line 205, in __call__
    await self.send_queue.join()
  File "/Users/r.slinckx/.pyenv/versions/3.12.1/lib/python3.12/asyncio/queues.py", line 215, in join
    await self._finished.wait()
  File "/Users/r.slinckx/.pyenv/versions/3.12.1/lib/python3.12/asyncio/locks.py", line 212, in wait
    await fut
asyncio.exceptions.CancelledError
Task exception was never retrieved
future: <Task finished name='Task-59' coro=<WSGIResponder.sender() done, defined at /path/.venv/lib/python3.12/site-packages/a2wsgi/wsgi.py:222> exception=ClientDisconnected()>
Traceback (most recent call last):
  File "/path/.venv/lib/python3.12/site-packages/a2wsgi/wsgi.py", line 228, in sender
    await send(message)
  File "/path/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 50, in sender
    await send(message)
  File "/path/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 161, in _send
    await send(message)
  File "/path/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 465, in send
    raise ClientDisconnected
uvicorn.protocols.utils.ClientDisconnected

The exception is something new in uvicorn 0.28, but it seems there's something wrong when handling it in a2wsgi

I haven't reproduced this problem in Uvicorn 0.28. Judging from the traceback, this is caused by the sender sending data to the client too slowly.

(I edited, There was a typo in my previous message (route /healhtz -> /healthz), but this shouldn't change the result)

The provided example does nothing particularly slow, and the while true does curl sequentially, so I would expect curl to wait until the response before starting a new connection, there's no concurrency in this example. It appears as if something in the handling of the response gets stuck somewhere. Just to be sure here are some more explicit command line arguments for uvicorn (although i get the same with --http h11 and --loop uvloop):
.venv/bin/uvicorn uvicornfail:app --loop asyncio --http httptools --limit-concurrency 2

Also this is with flask==3.0.2

I tried with modifying a2wsgi.wsgi.WSGIResponder.sender as follow:

    async def sender(self, send: Send) -> None:
        while True:
            message = await self.send_queue.get()
            self.send_queue.task_done()
            print(f'sender got message: {message!r}')
            if message is None:
                return
            from uvicorn.protocols.utils import ClientDisconnected
            try:
                await send(message)
            except ClientDisconnected:
                import traceback; traceback.print_exc()
                continue

Which gives the following logs:

Working request:
sender got message: {'type': 'http.response.body', 'body': b'OK', 'more_body': True}
sender got message: {'type': 'http.response.body', 'body': b''}
sender got message: None
sender got message: {'type': 'http.response.start', 'status': 200, 'headers': [(b'content-type', b'text/html; charset=utf-8'), (b'content-length', b'2')]}
INFO:     127.0.0.1:58246 - "GET /healthz HTTP/1.1" 200 OK

Weird request:
sender got message: {'type': 'http.response.body', 'body': b'OK', 'more_body': True}
sender got message: {'type': 'http.response.body', 'body': b''}
Traceback (most recent call last):
  File "/.venv/lib/python3.12/site-packages/a2wsgi/wsgi.py", line 231, in sender
    await send(message)
  File "/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 50, in sender
    await send(message)
  File "/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 161, in _send
    await send(message)
  File "/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 469, in send
    raise ClientDisconnected
uvicorn.protocols.utils.ClientDisconnected
sender got message: None
sender got message: {'type': 'http.response.start', 'status': 200, 'headers': [(b'content-type', b'text/html; charset=utf-8'), (b'content-length', b'2')]}

Due to adding an except: continue, the server keeps going now, but I can't explain why the client disconnects at that point

@rslinckx Thank you for the DEBUG information. It is true that a2wsgi should not trust send to be successful. I have released version 1.10.4, please try it again when you have time. (I can't reproduce it locally)

a2wsgi/a2wsgi/wsgi.py

Lines 192 to 213 in 8169f0a

    
           async def __call__(self, scope: HTTPScope, receive: Receive, send: Send) -> None: 
        
               body = Body(self.loop, receive) 
        
               environ = build_environ(scope, body) 
        
               sender = None 
        
               try: 
        
                   sender = self.loop.create_task(self.sender(send)) 
        
                   context = contextvars.copy_context() 
        
                   func = functools.partial(context.run, self.wsgi) 
        
                   await self.loop.run_in_executor( 
        
                       self.executor, func, environ, self.start_response 
        
                   ) 
        
                   await self.send_queue.put(None) 
        
                   # Sender may raise an exception, so we need to await it 
        
                   # dont await send_queue.join() because it will never finish 
        
                   await sender 
        
                   if self.exc_info is not None: 
        
                       raise self.exc_info[0].with_traceback( 
        
                           self.exc_info[1], self.exc_info[2] 
        
                       ) 
        
               finally: 
        
                   if sender and not sender.done(): 
        
                       sender.cancel()  # pragma: no cover

With the new release I can't reproduce the problem, and all requests are handled properly in the above code sample. I'm not entirely sure why the ClientDisconnected got triggered though, but that may be something else. In any case I don't see any visible problems. I'll let you know if it doesn't work properly in a more real environment when it is deployed.

Thanks a lot!

@rslinckx thanks for creating that example, I was struggling to create one that was minimal. @abersheeran I can confirm that my issue is fixed in version 1.10.4 as well.

	async def __call__(self, scope: HTTPScope, receive: Receive, send: Send) -> None:
	body = Body(self.loop, receive)
	environ = build_environ(scope, body)
	sender = None
	try:
	sender = self.loop.create_task(self.sender(send))
	context = contextvars.copy_context()
	func = functools.partial(context.run, self.wsgi)
	await self.loop.run_in_executor(
	self.executor, func, environ, self.start_response
	)
	await self.send_queue.put(None)
	# Sender may raise an exception, so we need to await it
	# dont await send_queue.join() because it will never finish
	await sender
	if self.exc_info is not None:
	raise self.exc_info[0].with_traceback(
	self.exc_info[1], self.exc_info[2]
	)
	finally:
	if sender and not sender.done():
	sender.cancel() # pragma: no cover

Unhandled error in ASGI middleware below wrapped WSGI app causes the app to hang.