graphql-java / java-dataloader

A Java 8 port of Facebook DataLoader

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ScheduledDataLoaderRegistry can leak scheduler threads

DanielThomas opened this issue · comments

Related to #135, I noticed that the executor isn't shut down, so if it's used, would leak the single core thread.

Since JDK 9 the default keepalive timeout is 10ms, so allowCoreThreadTimeOut is a potential solution, if you want to avoid explicit shutdown() in close():

https://github.com/openjdk/jdk/blob/caf71810f85ea55083ce7d1c76307a0c42d9be0e/src/java.base/share/classes/java/util/concurrent/ScheduledThreadPoolExecutor.java#L429-L443

Incidentally, folks on JDK 21 using JSON thread dumps should watch for this, because it's tricky to troubleshoot:

  • On JDK 21 ThreadPoolExecutors register as ThreadContainers, a concept added to support thread dumps with virtual threads
  • We happen have a periodic thread dump script, which takes JSON thread dumps
  • TheadContainers#children doesn't scale well, on my machine with 30K containers (2m of runtime @ 250 rps) ThreadDumper takes a minute
  • On a real application instance, that caused ThreadDumper to run indefinitely, because the application outpaces the ThreadDumper's ability to walk thread containers
  • ThreadDumper also dutifully collects the ThreadContainers onto a list, which caused the weak references to the containers to live forever, causing a memory leak

#144 addresses this