improve exception handling for load balancer
aniketmaurya opened this issue · comments
[root.load_balancer] 2022-09-27T11:16:30.861Z INFO: aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
[root.load_balancer] 2022-09-27T11:16:30.861Z ERROR: Exception in ASGI application
[root.load_balancer] 2022-09-27T11:16:30.887Z INFO: Traceback (most recent call last):
[root.load_balancer] 2022-09-27T11:16:30.887Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.888Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z raise e
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.888Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.888Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.888Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.888Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.888Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.888Z raise result
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.888Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.888Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.888Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z raise e
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.889Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.889Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.889Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.889Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.889Z raise result
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.889Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.889Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.889Z raise e
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.890Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.890Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.890Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.890Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.890Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.890Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.890Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.890Z raise result
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 42, in send_batch
[root.load_balancer] 2022-09-27T11:16:30.890Z async with session.post(f"{server}/api/predict", json=data, timeout=REQUEST_TIMEOUT) as result:
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/client.py", line 1141, in __aenter__
[root.load_balancer] 2022-09-27T11:16:30.890Z self._resp = await self._coro
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/client.py", line 560, in _request
[root.load_balancer] 2022-09-27T11:16:30.890Z await resp.start(conn)
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 899, in start
[root.load_balancer] 2022-09-27T11:16:30.890Z message, payload = await protocol.read() # type: ignore[union-attr]
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
[root.load_balancer] 2022-09-27T11:16:30.890Z await self._waiter
[root.load_balancer] 2022-09-27T11:16:30.890Z INFO: aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
[root.load_balancer] 2022-09-27T11:16:30.890Z ERROR: Exception in ASGI application
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: Traceback (most recent call last):
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.891Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z raise e
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.891Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.891Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.891Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.891Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.892Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.892Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.892Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.892Z raise result
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.892Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z raise e
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.892Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.892Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.892Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.892Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.892Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.893Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.893Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.893Z raise result
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.893Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z raise e
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.893Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.893Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.893Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.893Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.893Z raise result
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
[root.load_balancer] 2022-09-27T11:16:30.893Z result = await app( # type: ignore[func-returns-value]
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z return await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/applications.py", line 270, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await super().__call__(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z await self.middleware_stack(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.893Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
[root.load_balancer] 2022-09-27T11:16:30.893Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z await self.app(scope, receive, _send)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 84, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z raise exc
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z await self.app(scope, receive, sender)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z raise e
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 680, in __call__
[root.load_balancer] 2022-09-27T11:16:30.894Z await route.handle(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 275, in handle
[root.load_balancer] 2022-09-27T11:16:30.894Z await self.app(scope, receive, send)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
[root.load_balancer] 2022-09-27T11:16:30.894Z response = await func(request)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 231, in app
[root.load_balancer] 2022-09-27T11:16:30.894Z raw_response = await run_endpoint_function(
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
[root.load_balancer] 2022-09-27T11:16:30.894Z return await dependant.call(**values)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 159, in balance_api
[root.load_balancer] 2022-09-27T11:16:30.894Z return await self.process_request(data)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 197, in process_request
[root.load_balancer] 2022-09-27T11:16:30.894Z raise result
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/muse/components/load_balancer.py", line 42, in send_batch
[root.load_balancer] 2022-09-27T11:16:30.894Z async with session.post(f"{server}/api/predict", json=data, timeout=REQUEST_TIMEOUT) as result:
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/client.py", line 1141, in __aenter__
[root.load_balancer] 2022-09-27T11:16:30.894Z self._resp = await self._coro
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/client.py", line 560, in _request
[root.load_balancer] 2022-09-27T11:16:30.894Z await resp.start(conn)
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 899, in start
[root.load_balancer] 2022-09-27T11:16:30.894Z message, payload = await protocol.read() # type: ignore[union-attr]
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: File "/home/zeus/.local/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
[root.load_balancer] 2022-09-27T11:16:30.894Z await self._waiter
[root.load_balancer] 2022-09-27T11:16:30.894Z INFO: aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
[root.load_balancer] 2022-09-27T11:16:30.894Z ERROR: Exception in ASGI application
this was fixed