Error "unable to load native libsodium" but the system does have it

Question

Error "unable to load native libsodium" but the system does have it

mtmr0x opened this issue 3 years ago · comments

I've trying to run a backend API that uses Caesium (version "0.14.0") in my MacOS with M1 chip and I'm getting the following error:

Execution error (ClassNotFoundException) at caesium.binding/load-sodium (binding.clj:716).
unable to load native libsodium; is it installed?

Full error attached below

I've tried to run it with a diversity of JDKs but I keep getting the exact same error. In non M1 chips the project runs with no issues (with some exceptions on Macs that require to use JDK 11).

I could isolate the problem to Caesium, considering the project is using more cryptographic technologies I wanted to verify if any other item were using it.

Removing Caesium from use the project runs.

Here are the JDK I tried, using lein with brew on openjdk and manually installing lein and using jabba to manage the JDK version on zulu.

release	pass?
zulu@1.11.0	❌
zulu@1.16.0	❌
adoptopenjdk@16.0	❌
adoptopenjdk@14.0	❌
adoptopenjdk@11.0	❌

FULL ERROR MESSAGE HERE, click to expand

{:clojure.main/message
"Execution error (ClassNotFoundException) at caesium.binding/load-sodium (binding.clj:716).\nunable to load native libsodium; is it installed?\n",
:clojure.main/triage
{:clojure.error/class java.lang.ClassNotFoundException,
:clojure.error/line 716,
:clojure.error/cause
"unable to load native libsodium; is it installed?",
:clojure.error/symbol caesium.binding/load-sodium,
:clojure.error/source "binding.clj",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type clojure.lang.Compiler$CompilerException,
  :message "Syntax error macroexpanding at (binding.clj:722:3).",
  :data
  {:clojure.error/phase :execution,
   :clojure.error/line 722,
   :clojure.error/column 3,
   :clojure.error/source "binding.clj"},
  :at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3707]}
 {:type java.lang.ClassNotFoundException,
  :message "unable to load native libsodium; is it installed?",
  :at [caesium.binding$load_sodium invokeStatic "binding.clj" 716]}],
:trace
[[caesium.binding$load_sodium invokeStatic "binding.clj" 716]
 [caesium.binding$load_sodium invoke "binding.clj" 707]
 [caesium.binding$load_sodium invokeStatic "binding.clj" 710]
 [caesium.binding$load_sodium invoke "binding.clj" 707]
 [clojure.lang.AFn applyToHelper "AFn.java" 152]
 [clojure.lang.AFn applyTo "AFn.java" 144]
 [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3702]
 [clojure.lang.Compiler$DefExpr eval "Compiler.java" 457]
 [clojure.lang.Compiler eval "Compiler.java" 7182]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 436]
 [caesium.crypto.box$eval26648$loading__6721__auto____26649
  invoke
  "box.clj"
  1]
 [caesium.crypto.box$eval26648 invokeStatic "box.clj" 1]
 [caesium.crypto.box$eval26648 invoke "box.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7166]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 482]
 [security.crypto$eval26640$loading__6721__auto____26641
  invoke
  "crypto.clj"
  1]
 [security.crypto$eval26640 invokeStatic "crypto.clj" 1]
 [security.crypto$eval26640 invoke "crypto.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7166]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 703]
 [security.login.login$eval17253$loading__6721__auto____17254
  invoke
  "login.clj"
  1]
 [security.login.login$eval17253 invokeStatic "login.clj" 1]
 [security.login.login$eval17253 invoke "login.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7166]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 457]
 [security.login.interceptor$eval17247$loading__6721__auto____17248
  invoke
  "interceptor.clj"
  1]
 [security.login.interceptor$eval17247
  invokeStatic
  "interceptor.clj"
  1]
 [security.login.interceptor$eval17247 invoke "interceptor.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7166]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 930]
 [http.routes$eval16978$loading__6721__auto____16979
  invoke
  "routes.clj"
  1]
 [http.routes$eval16978 invokeStatic "routes.clj" 1]
 [http.routes$eval16978 invoke "routes.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7166]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 482]
 [http.server$eval157$loading__6721__auto____158
  invoke
  "server.clj"
  1]
 [http.server$eval157 invokeStatic "server.clj" 1]
 [http.server$eval157 invoke "server.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7166]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.RT loadResourceScript "RT.java" 381]
 [clojure.lang.RT loadResourceScript "RT.java" 372]
 [clojure.lang.RT load "RT.java" 459]
 [clojure.lang.RT load "RT.java" 424]
 [clojure.core$load$fn__6839 invoke "core.clj" 6126]
 [clojure.core$load invokeStatic "core.clj" 6125]
 [clojure.core$load doInvoke "core.clj" 6109]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5908]
 [clojure.core$load_one invoke "core.clj" 5903]
 [clojure.core$load_lib$fn__6780 invoke "core.clj" 5948]
 [clojure.core$load_lib invokeStatic "core.clj" 5947]
 [clojure.core$load_lib doInvoke "core.clj" 5928]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$load_libs invokeStatic "core.clj" 5985]
 [clojure.core$load_libs doInvoke "core.clj" 5969]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 667]
 [clojure.core$require invokeStatic "core.clj" 6007]
 [clojure.core$require doInvoke "core.clj" 6007]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [user$eval140$fn__144 invoke "form-init10770987251725177174.clj" 1]
 [user$eval140 invokeStatic "form-init10770987251725177174.clj" 1]
 [user$eval140 invoke "form-init10770987251725177174.clj" 1]
 [clojure.lang.Compiler eval "Compiler.java" 7177]
 [clojure.lang.Compiler eval "Compiler.java" 7167]
 [clojure.lang.Compiler load "Compiler.java" 7636]
 [clojure.lang.Compiler loadFile "Compiler.java" 7574]
 [clojure.main$load_script invokeStatic "main.clj" 475]
 [clojure.main$init_opt invokeStatic "main.clj" 477]
 [clojure.main$init_opt invoke "main.clj" 477]
 [clojure.main$initialize invokeStatic "main.clj" 508]
 [clojure.main$null_opt invokeStatic "main.clj" 542]
 [clojure.main$null_opt invoke "main.clj" 539]
 [clojure.main$main invokeStatic "main.clj" 664]
 [clojure.main$main doInvoke "main.clj" 616]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.lang.Var applyTo "Var.java" 705]
 [clojure.main main "main.java" 40]],
:cause "unable to load native libsodium; is it installed?",
:phase :execution}}

If anyone have a solution or workaround to run the project locally in my machine I would really appreciate it. Thanks

Dominic Pearson · Answer 1 · Sat Jul 31 2021 02:49:57 GMT+0800 (China Standard Time)

What is the full path to your libsodium library? Is it in your java.library.path? This is probably either an architecture mismatch or a JVM library path issue. Running (System/getProperty "java.library.path") at a REPL should show you where the default search path is for the JVM. I believe libsodium needs to be in one of the listed directories to be picked up by JNR-FFI. I don't have a M1 to try on, but more information would be great for trying to pin down the issue.

Sandro Mello · Answer 2 · Tue Aug 10 2021 03:13:24 GMT+0800 (China Standard Time)

I've followed the tip of @dspearson and linked the libsodium library to a path in java.library.path and it worked:

# this may differ depending of each O.S.
sudo ln -s /path/to/libsodium/shared/library/libsodium.dylib /path/to/java/library/libsodium.dylib

My setup is a mac-mini amd64

Matheus Marsiglio · Answer 3 · Wed Aug 11 2021 02:13:33 GMT+0800 (China Standard Time)

I just did (apparently) what @sandromello did, but it didn't solve for me.

Based on your message @dspearson I ran (System/getProperty "java.library.path") from my REPL and got:

"/Users/mtmr0x/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:."

I've run a symlink like this (since the path of /Library/Java/Extensions were the only valid/that existed in my system):

sudo ln -s /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib /Library/Java/Extensions/libsodium.dylib

Is there something I'm missing? One thing that's a bit confusing is the : between every possible path from (System/getProperty "java.library.path") that I got in the REPL.

Context: I'm quite new to Java/Clojure. Some basic conventions might be off my knowledge

Dominic Pearson · Answer 4 · Thu Aug 12 2021 02:27:01 GMT+0800 (China Standard Time)

The : is conventionally a separator on UNIX, so each of those paths will be searched when trying to load libs. That should have worked. You could try cp /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib /Users/mtmr0x/Library/Java/Extensions, in case it is somehow a permission issue, though I am not sure I can be of much further help since I don't have access to macOS. Maybe someone else could further assist.

胡雨軒 Петр · Answer 5 · Thu Dec 16 2021 21:32:03 GMT+0800 (China Standard Time)

I can confirm that I was able to get it working in the REPL, but I got issues with lein install. I executed

ln -s /usr/local/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib $HOME/Library/Java/Extensions/libsodium.dylib
lein clean

and now lein install works smoothly.

Dominic Pearson · Answer 6 · Thu Jan 13 2022 16:16:14 GMT+0800 (China Standard Time)

Great news! A macOS/brew library path idiosyncrasy then, which would affect any JVM code that calls native code outside of known paths, rather than a bug with caesium. There's not really any good way to resolve this within the project. If anything, it'd be upstream brew and java defaults. I guess the issue here can be closed?

Matheus Marsiglio · Answer 7 · Thu Jan 13 2022 21:33:49 GMT+0800 (China Standard Time)

Great news! A macOS/brew library path idiosyncrasy then, which would affect any JVM code that calls native code outside of known paths, rather than a bug with caesium. There's not really any good way to resolve this within the project. If anything, it'd be upstream brew and java defaults. I guess the issue here can be closed?

Done! ✌️