Rewrite GraphQL Subscriptions to django-channels-graphql-ws

Question

pziemkowski opened this issue 5 months ago · comments

Current implementation of GraphQL subscriptions is very hacky and not scalable at all (even though that is the main reason for its hackiness).

Here's a breakdown of implementation:

AWS API Gateway is used as a connection backend for web sockets
AWS API Gateway runs Lambda functions when connection is established, lost, and message is received
Each message causes a request to our backend API, through regular load balancer to one of running AWS ECS tasks
GraphQL subscription query is written to DB
When an event that should cause execution of GraphQL query occurs all of matching GraphQL queries are executed in the very same AWS ECS task and dispatched back to the connected client, again through API Gateway.

You can clearly see that even though we wanted to achieve scalability by moving web socket connections outside our AWS ECS we caused another, even harsher performance degradation – all GraphQL subscription execution run synchronously in the same ECS task (instance of the app).
AWS API Gateway cannot be run on local machine, so we develop a very hacky solution – local ws server, which is a simulator of API Gateway + Lambda functions. That is very hard to maintain, as well as very hard to understand for other developers.
Our implementation of query execution is quite crude and AFAIK doesn't support permission checks.

My suggestion is to use django-channels-graphql-ws package which solves all of the problems above, but cames with some drawbacks:

we manage ws connections directly through Load Balancer on the ECS task with API Backend. Something that generally pushed us into the current solution.
we need to add Redis to our stack, but that is most likely happening anyway in near future for other features we have planned.

In my opinion though the advantages far outweigh the disadvantages.

No response

No response

Follow our Code of Conduct.
Read the Contributing Guidelines.
Read the docs.
Check that there isn't already an issue that request the same feature to avoid creating a duplicate.