(bug) Install on macOS arm64 comes with x86_64 native libraries
Prong4107 opened this issue · comments
Describe the problem
When installing jep using a venv on a arm64 mac with pip, native libraries results are for the wrong CPU architecture, resulting in a crash when trying to load libjep.jnilib
**Environment
Hardware Overview:
Model Name: Mac mini
Model Identifier: Macmini9,1
Model Number: MGNR3LL/A
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 8 GB
System Firmware Version: 8422.141.2
OS Loader Version: 8422.141.2
Serial Number (system): H2WDR39CQ6NV
Hardware UUID: 9D7A7B24-F69B-5407-AB83-45C4115605B5
Provisioning UDID: 00008103-000664A01840291E
Activation Lock Status: Disabled
Hardware Overview:
Model Name: Mac mini
Model Identifier: Macmini9,1
Model Number: MGNR3LL/A
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 8 GB
System Firmware Version: 8422.141.2
OS Loader Version: 8422.141.2
Serial Number (system): H2WDR39CQ6NV
Hardware UUID: 9D7A7B24-F69B-5407-AB83-45C4115605B5
Provisioning UDID: 00008103-000664A01840291E
Activation Lock Status: Disabled
$ uname -a
Darwin EU852 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:52 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T8103 arm64
Python 3.11.5 installed with homebrew
$ java --version
openjdk 11.0.16.1 2022-08-12 LTS
OpenJDK Runtime Environment Microsoft-40648 (build 11.0.16.1+1-LTS)
OpenJDK 64-Bit Server VM Microsoft-40648 (build 11.0.16.1+1-LTS, mixed mode)
Reproduction
$ python3 --version
Python 3.11.5
$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install jep
Collecting jep
Using cached jep-4.1.1-cp311-cp311-macosx_13_0_arm64.whl
Installing collected packages: jep
Successfully installed jep-4.1.1
(venv) $ file venv/lib/python3.11/site-packages/jep/libjep.jnilib
venv/lib/python3.11/site-packages/jep/libjep.jnilib: Mach-O 64-bit bundle x86_64
(venv) $ file venv/lib/python3.11/site-packages/jep/
(venv) $ file venv/lib/python3.11/site-packages/jep/jep.cpython-311-darwin.so
venv/lib/python3.11/site-packages/jep/jep.cpython-311-darwin.so: Mach-O 64-bit bundle x86_64
(venv) $ jep
java.lang.UnsatisfiedLinkError: Can't load library: /Users/user934234/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/jep/libjep.jnilib
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2633)
at java.base/java.lang.Runtime.load0(Runtime.java:768)
at java.base/java.lang.System.load(System.java:1837)
at jep.LibraryLocator.searchPackageDir(LibraryLocator.java:243)
at jep.LibraryLocator.searchSitePackages(LibraryLocator.java:146)
at jep.LibraryLocator.findJepLibrary(LibraryLocator.java:325)
at jep.MainInterpreter.initialize(MainInterpreter.java:131)
at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:101)
at jep.Jep.<init>(Jep.java:133)
at jep.SharedInterpreter.<init>(SharedInterpreter.java:60)
at jep.Run.run(Run.java:49)
at jep.Run.main(Run.java:146)
Please ensure that your Java and Python are both using the same architecture. The jep library is built to match your python architecture so it appears you are mixing an x86_64 python with an arm64 Java which is not supported.
If you are having trouble installing compatible versions I recommend reviewing the previous discussion in #379, #389, #423, and #442.
Actually, it seems my problem was that I installed jep in x86_64 mode a first time, and pip cached the build, so when I installed jep in a venv a second time in arm64, it used the cached x86_64 binaries instead of rebuilding for arm64.
Running pip cache purge before pip install seems to have solved it.
Not entirely sure how the inconsistencies in architectures first happened though.
Thanks for the help.