Update to Java 11 breaks Pyserini
chriskamphuis opened this issue · comments
When updating to Java 11 the path the JRE is not correctly configured for latest version of pyjnius
that is installed with pip
. This can be fixed by cloning the latest version from github and installing that version, but I assume this undesirable.
hey @chriskamphuis any idea why src/test/python/python_interface.py
didn't fail then? it's part of Travis CI definition...
no clue .... it did not work for me locally because of this, so I assumed it was a problem in general, maybe it was my system specifically
btw your Travis uses Python 2. Dont think that will be the reason, but might be nice to change it to Python 3.
A bit more detail on this; I've replicated this bug. On macOS 10.14.6:
(python36) iMac-Pro:anserini jimmylin$ conda list | grep pyjnius
pyjnius 1.2.0 <pip>
The error I get is:
>>> from pyserini.search import pysearch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "./src/main/python/pyserini/search/pysearch.py", line 21, in <module>
from ..pyclass import JSearcher, JString, JArrayList
File "./src/main/python/pyserini/pyclass.py", line 28, in <module>
from jnius import autoclass, cast
File "/anaconda3/envs/python36/lib/python3.6/site-packages/jnius/__init__.py", line 13, in <module>
from .reflect import * # noqa
File "/anaconda3/envs/python36/lib/python3.6/site-packages/jnius/reflect.py", line 15, in <module>
class Class(with_metaclass(MetaJavaClass, JavaClass)):
File "/anaconda3/envs/python36/lib/python3.6/site-packages/six.py", line 827, in __new__
return meta(name, bases, d)
File "jnius/jnius_export_class.pxi", line 114, in jnius.MetaJavaClass.__new__
File "jnius/jnius_export_class.pxi", line 164, in jnius.MetaJavaClass.resolve_class
File "jnius/jnius_env.pxi", line 11, in jnius.get_jnienv
File "jnius/jnius_jvm_dlopen.pxi", line 90, in jnius.get_platform_jnienv
File "jnius/jnius_jvm_dlopen.pxi", line 59, in jnius.create_jnienv
SystemError: Error calling dlopen(b'/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib': b'dlopen(/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib, 10): image not found'
Issue is documented here.
Indeed, the lib has moved:
$ ls /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib
/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib
So a (janky) symlink will fix the issue:
$ sudo mkdir -p /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/
$ ls /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib
/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib
$ sudo ln -s /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib
The colab demo doesn't seem to have this issue: https://colab.research.google.com/drive/1GOCyWyYW_fwJMKy5FVWdYHEC2IlvpqZN
Edit: nevermind, above colab notebook doesn't use pyjnius.
I just noticed, @Impavidity and @tuzhucheng mentioned the same issue (but different OS?) in #805. Unclear why colab works though...?
Colab demo illustrating issue and fix: https://colab.research.google.com/drive/1r1pRq_BfWS486kg2qwVH5iBfbhK_GPCg
Issue and resolution are documented in https://github.com/castorini/pyserini
Closing issue.