Use a permanent session with Django?
markedwards opened this issue · comments
I'm struggling to make the permanent-session example work with Django. I tried a singleton approach with an async getter, which creates the client and reconnecting session on first access. This works for one request, but I get Event loop is closed errors on subsequent requests.
Is there a way to make this work, or is it only possible to share a session for a given request with Django's architecture?
My guess is the answer is it's only possible if Django is combined with ASGI. Otherwise, I suppose its limited to a session per-request.
Did you run the asyncio event loop in a separated thread?
No, but the intention is for this to be used in DataLoaders inside GraphQL resolvers, via a Django view. And all of those run in the Django request loop, I suppose.
Well I don't know anything about django but I gave it a try with the beginning of the polls tutorial.
If you create a new event loop and put the permanent gql connection in it, then you can execute coroutine on that loop by running asyncio.run_coroutine_threadsafe.
In that way you don't use the django event loop at all and the gql stuff is contained completely in a separate thread.
It might not be the perfect way to structure this in django but here is what I have:
polls/apps.py:
from django.apps import AppConfig
import asyncio
import threading
from .gql_client import GraphQLContinentClient
def gql_background_task(gql_client, gql_event_loop):
gql_event_loop.run_until_complete(gql_client.connect())
gql_event_loop.run_forever()
class PollsConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "polls"
def ready(self):
self.gql_client = GraphQLContinentClient()
self.gql_event_loop = asyncio.new_event_loop()
self.gql_thread = threading.Thread(
target=gql_background_task,
args=(self.gql_client, self.gql_event_loop)
)
self.gql_thread.start()
def get_continent(self, continent_code):
loop = self.gql_event_loop
coro = self.gql_client.get_continent_name(continent_code)
result_future = asyncio.run_coroutine_threadsafe(coro, loop)
return result_future.result() polls/views.py:
from django.shortcuts import render
from django.apps import apps
from django.http import HttpResponse
def index(request):
app_config = apps.get_app_config('polls')
eu_name = app_config.get_continent("EU")
return HttpResponse(f"The EU continent name is {eu_name}") polls/gql_client.py:
from gql.transport.aiohttp import AIOHTTPTransport
from gql import Client, gql
GET_CONTINENT_NAME = """
query getContinentName ($code: ID!) {
continent (code: $code) {
name
}
}
"""
class GraphQLContinentClient:
def __init__(self):
self._client = Client(
transport=AIOHTTPTransport(url="https://countries.trevorblades.com/")
)
self._session = None
self.get_continent_name_query = gql(GET_CONTINENT_NAME)
async def connect(self):
self._session = await self._client.connect_async(reconnecting=True)
async def close(self):
await self._client.close_async()
async def get_continent_name(self, code):
print ("Getting continent name")
params = {"code": code}
answer = await self._session.execute(
self.get_continent_name_query, variable_values=params
)
return answer.get("continent").get("name") and in mysite/settings.py:
INSTALLED_APPS = [
...
"polls.apps.PollsConfig",
] I didn't really explain my use-case very well. I have an async view running under ASGI (so each request runs in a loop which only lives exists for that request). The view is itself providing a GraphQL endpoint, with a complex maze of async child resolver functions. For a given request, those resolver functions need to be able to independently await multiple calls to an external GraphQL endpoint via gql.
It seems to me that the best that can be achieved in this scenario would be to create a gql session for each Django request (rather than for each external call as I'm currently doing it), and use that for the duration of the request. I don't see how it's possible to share an application-wide gql session effectively in this setup, without switching to ASGI.
The point of the permanent session is to have a long-lived gql session which can automatically reconnect in case of connection failure. I'm not sure your really need this in your case (for each Django request).
Anyway, there is a lot of possible ways to structure your application for your needs and it's difficult to help you further without knowing your constraints. I've shown how you can have a permanent session (running forever) in a separate event loop running in a separate thread which allows you to execute queries even from sync methods, WSGI or ASGI, it does not matter.
A permanent session for each Django request should be possible also. You could use the same trick as I did above but that might be overkill and not smart performance-wise to create a new thread and a new asyncio event loop for each Django request.
Please post a minimal reproducible example that I should be able to run quickly to show where your problem lies and I might be able to help you further.
It's okay, I think I've worked my way through it. Sorry to take up space here investigating my own lack of knowledge. :-)
Yeah, I think it's not a smart design to do a permanent session in the WSGI scenario that I have. I think it will only make things slower. In an ASGI context, there is one global loop, so following the stock example from the gql docs should work there (unless I'm missing something).
My thinking in terms of a per-request session being worth it was that there could be some benefit to reuse of the session, since there are N gql executions per request (where N can be, say, a dozen). But perhaps there's no benefit in this scenario over simply spawning a new client/session for each gql execution.
Closing for now. I'll reopen if there's a concrete example to discuss. Thanks!
Alright! See also issue #381 if you want to manage the TCP connection yourself.
That way the TCP connection is not closed between gql sessions.