Multithreaded python calls to java return deranged results

[sagewanglun]

Background: Start with the java code and use gunicorn to start the python server with multiple works

The java side uses the following code:
public static void main(String[] args) {
GatewayServer server = new GatewayServer(func);
server.start();

The python side uses the following code:
from py4j.java_gateway import JavaGateway
jvm = JavaGateway()
function = jvm.getfunc()
When multiple threads from python call java, the result will be confused. For example, the input of thread 1 is not returned to thread 1, but to thread 2

[Hyukjin Kwon]

Does it use multiprocessing from gunicorn? Py4J is thread-safe but doesn't work well with multiprocessing.

[sagewanglun]

When will this problem be solved？

[Hyukjin Kwon]

If this is from multiprocessing, it would not be fixed because Py4J does not support multiprocessing by design.

[sagewanglun]

I tried using gunicorn to start a work to start a python method, that is, a single process, and then the python method used multiple threads to call java, but it still returned confusing results. Am I doing it wrong?

[Hyukjin Kwon]

Would be great if there's a reproducer without gunicorn to investigate this further. My guts say gurnicorn uses a separate process when you use Py4J.