Silent failure of jpype.startJVM when classpath is passed - RESOLVED

[Ian Hopkinson]

This is more a comment than an issue, feel free to close without response. I place it here to assist anyone else having a similar problem, and to understand why my fix works, if possible.

A couple of weeks ago some code on my Windows 10 laptop which used JPype stopped working. This invocation stopped working, providing the classpath in the startJVM call:

import jpype
jpype.startJVM(classpath=[os.path.join(CUSTOM_PATH, 'custom.jar')])

The mode of failure, when invoked from ipython, was simply to silently exit from ipython back to the shell prompt - without the classpath argument the behaviour is simply to return control quickly to the ipython prompt. startJVM continued to work if no classpath argument was supplied.

After considerable fiddling with my Java settings, multiple reinstalls of JPype (including a build with tracing enabled), upgrading my Anaconda distribution. The solution turned out to be to change the invocation to this, i.e. updating the classpath once the JVM has started:

import jpype
jpype.startJVM()
jpype.addClassPath(os.path.join(CUSTOM_PATH, 'custom.jar'))

I don't know what changed on my laptop to stop JPype working. It coincided roughly in time with my IT department alerting me to an unpatched version of log4j appearing on my system in the package cache. I deleted this directly and disabled various random Java related services - I use VS Code and the Java Extension Pack seems to do a lot of beavering away in the background. But the error persisted after a re-install of the JDK.

Any comment as to why this fix should work, and what might have caused the issue in the first place would be welcome (but not worth in depth investigation on your part).

[Karl Nelson]

I don't have a good explaination for this failure. The JVM does act differently depending on how it is invoked. In the later form the JAR is actually loaded into a special class loader which acts a bit differently. Unfortunately when loaded later the JAR lack certain privileges such as acting as a database driver which is why it is not the preferred method.

What should happen the case of a failed load is the jvm should have written a crash log. If there was not crash log then the next most likely is that the JAR itself invoked the "exit()" which will cause the Python process to terminate. Though I have no clue as to why a jar would do such a thing. Unfortunately without a log to determine what code was executed that caused the failure I can't fathom how one would trace this down.

My first recommendation is run JPype with trace enabled. That should show you if it crashed in the startJVM in the private module or after the JVM was started during resource allocation. The next suggestion is rather horrible but it is the method that I have sometimes used which is to write a javaagent which hooks to the classloader methods. You can then find out what code was loaded just prior to the crash.

[John Hennig]

As far as I understand the documentation, calling jpype.addClassPath() after jpype.startJVM() simply has no effect:

One should note that the class path can only be set prior starting the JVM. Calls to set the class path after the JVM is started are silently ignored. If a jar must be loaded after the JVM is started, it may be loaded using java.net.URLClassLoader.

So the custom.jar should not have been loaded at all in the above case. This lines up with my own experience. And would explain why there's no error then. Though it would also not solve the actual problem.

[Ian Hopkinson]

@Thrameos thanks for your comments. I should also say I really appreciate your work on this library - it has been very useful for me. I ran JPype with tracing enabled, as I recall the last message before the crash was "startedJVM". There's no sign of a crash log in the immediate vicinity of the code. I don't have the Java skills to investigate further. My suspicion is that, since this is on a corporate laptop, some element of the Java system has been reconfigured centrally and that is the root cause.

@john-hen you are right, the documentation does imply that adding to the classpath after the JVM has been started has no effect! However, I had reached the stage where I was trying anything to get this to work again, and there is an issue elsewhere here which implies it does have an effect. It turns out that I can access functions from custom.jar after adding it to the classpath post-JVM start.

(I'm happy to close this issue, like I say I primarily posted as information for others)

[Karl Nelson]

I beleive that docmentation is out of date. We added dynamic classloader to save the user from this problem.