Deadlock when using subscriptions with two Ktor worker threads
micke-humla opened this issue · comments
Library Version
7.0.2
Describe the bug
A deadlock occurs when calling subscriptions in graphql-kotlin-ktor-server
if a suspendable function is called (that actually suspends execution) when there are only two workers (workerGroupSize
is set to 2 in application.conf
or using default settings on a machine with 2 CPUs). Tested with Netty as engine.
Explicitly setting workerGroupSize
to 3 or higher seems to fix the problem but might not be optimal on machines with only 2 CPUs. Problem surfaced on AWS Elastic Container Service with ECS instance reporting to have 2 CPUs (even with different vCPU values).
To Reproduce
I created a minimal reproducible example here and this is how it can be reproduced:
- Start the application with
./gradlew run
- Connect to the websocket at
ws://localhost:8080/subscriptions
- Send init message:
{"type":"connection_init"}
- Send subscribe message:
{"type":"subscribe", "id":"12345", "payload": {"query": "subscription { random }" } }
- No messages are returned and subscription "hangs" and the worker thread (
eventLoopGroupProxy
) has stateWAITING
Expected behavior
Subscriptions work even with "only" two worker threads
Hello 👋
I'm guessing the underlying issue is probably the same as with the #1898. If you change GraphQLServer
to switch threads for processing requests then it should work even if your Ktor engine starts with just 2 worker threads.
class MyKtorGraphQLServer(
requestParser: KtorGraphQLRequestParser,
contextFactory: KtorGraphQLContextFactory,
requestHandler: GraphQLRequestHandler
) : KtorGraphQLServer(requestParser, contextFactory, requestHandler) {
override suspend fun execute(request: ServerRequest): GraphQLServerResponse? = withContext(Dispatchers.Default) {
super.execute(request)
}
}
Thanks for the reply! Looks promising but unfortunately I find no way to override KtorGraphQLServer
(without also creating my custom com.expediagroup.graphql.server.ktor.GraphQL
class with all it's logic) so I can't verify if it solves the issue.
Also, would that really solve the problem for subscriptions which is using KtorGraphQLWebSocketServer
(which is also not possible to override)?
Thanks for the reply! Looks promising but unfortunately I find no way to override KtorGraphQLServer (without also creating my custom com.expediagroup.graphql.server.ktor.GraphQL class with all it's logic) so I can't verify if it solves the issue.
Good callout. I'll open up a separate issue to allow folks specifying their own GraphQL server impls.
Opened up #1902