Giters

terl / lazysodium-java

A Java implementation of the Libsodium crypto library. For the lazy dev.

Home Page:https://github.com/terl/lazysodium-java/wiki

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Resource Loading Error With Fat Jar

LukeHornibrookHaventec opened this issue · comments

commented

I'm encountering an error loading the bundled libsodium library using lazysodium 4.2.6 (and resource-loader 1.3.7) deployed within a fat jar. Here is a partially redacted stack trace:

Caused by: co.libly.resourceloader.ResourceLoaderException: Failed to load the bundled library from resources by relative path (linux64/libsodium.so),
at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:65),
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadBundledLibrary(LibraryLoader.java:133),
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadLibrary(LibraryLoader.java:106),
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method),
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62),
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45),
at java.lang.reflect.Constructor.newInstance(Constructor.java:423),
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:204),
... 27 common frames omitted,
Caused by: java.io.IOException: Failed to list contents of file:/opt/<path to the fat jar>.jar!/BOOT-INF/lib/lazysodium-java-4.2.6.jar!/linux64/libsodium.so,
at co.libly.resourceloader.ResourceLoader.doCopyDirectory(ResourceLoader.java:258),
at co.libly.resourceloader.ResourceLoader.copyDirectory(ResourceLoader.java:250),
at co.libly.resourceloader.ResourceLoader.getFileFromFileSystem(ResourceLoader.java:144),
at co.libly.resourceloader.ResourceLoader.copyToTempDirectory(ResourceLoader.java:89),
at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:52),
... 36 common frames omitted

After testing locally, ResourceLoader.getThisJarPath() returns a value such as jar:file:/opt/<path to the fat jar>.jar!/BOOT-INF/lib/lazysodium-java-4.2.6.jar!/. This in turn leads to a NoSuchFileException when loading the JarFile in ResourceLoader.isJarFile().

As an alternative approach to manually processing the jar file, can we instead use Thread.currentThread().getContextClassLoader().getResourceAsStream() to read the bundled libsodium` resource directly?

commented

Loading any type of resource file even simple JSON files from a JAR is messy business.

I can't remember exactly, but the reason why we don't use Thread.currentThread().getContextClassLoader().getResourceAsStream() was something to do with it working when retrieving things out of a JAR but when it came to loading things out of a regular directory it returned null. The safest way I have found was to copy the shared library to a temporary directory and load it from there.

I just tried this on my Mac and it all works when loaded from a far JAR in BUNDLED mode. I am not sure why it's not working in a linux environment, I need to debug further.

commented

This might be fixed because I migrated to AzureCI (#77) and identified some issues in an isolated environment. Please reopen if not fixed.

commented

@gurpreet- I still have issue.

I've created minimal sample project here https://github.com/kakawait/lazysodium-fatjar-issue

You can simply do

mvn clean package -DskipTests
java -jar target/demo-0.0.1-SNAPSHOT.jar

to reproduce and you should see

java.lang.IllegalStateException: Failed to execute CommandLineRunner
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) ~[spring-boot-2.3.4.RELEASE.jar!/:2.3.4.RELEASE]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:779) ~[spring-boot-2.3.4.RELEASE.jar!/:2.3.4.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) ~[spring-boot-2.3.4.RELEASE.jar!/:2.3.4.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) ~[spring-boot-2.3.4.RELEASE.jar!/:2.3.4.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.3.4.RELEASE.jar!/:2.3.4.RELEASE]
	at com.example.demo.DemoApplication.main(DemoApplication.java:14) ~[classes!/:0.0.1-SNAPSHOT]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:107) ~[demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[demo-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
Caused by: co.libly.resourceloader.ResourceLoaderException: Failed to load the bundled library from resources by relative path (mac/libsodium.dylib)
	at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:65) ~[resource-loader-1.3.7.jar!/:na]
	at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadBundledLibrary(LibraryLoader.java:133) ~[lazysodium-java-4.3.0.jar!/:na]
	at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadLibrary(LibraryLoader.java:94) ~[lazysodium-java-4.3.0.jar!/:na]
	at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:34) ~[lazysodium-java-4.3.0.jar!/:na]
	at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:23) ~[lazysodium-java-4.3.0.jar!/:na]
	at com.example.demo.DemoApplication$CommandLineAppStartupRunner.run(DemoApplication.java:22) ~[classes!/:0.0.1-SNAPSHOT]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) ~[spring-boot-2.3.4.RELEASE.jar!/:2.3.4.RELEASE]
	... 13 common frames omitted
Caused by: java.io.IOException: Failed to list contents of file:/tmp/demo/target/demo-0.0.1-SNAPSHOT.jar!/BOOT-INF/lib/lazysodium-java-4.3.0.jar!/mac/libsodium.dylib
	at co.libly.resourceloader.ResourceLoader.doCopyDirectory(ResourceLoader.java:258) ~[resource-loader-1.3.7.jar!/:na]
	at co.libly.resourceloader.ResourceLoader.copyDirectory(ResourceLoader.java:250) ~[resource-loader-1.3.7.jar!/:na]
	at co.libly.resourceloader.ResourceLoader.getFileFromFileSystem(ResourceLoader.java:144) ~[resource-loader-1.3.7.jar!/:na]
	at co.libly.resourceloader.ResourceLoader.copyToTempDirectory(ResourceLoader.java:89) ~[resource-loader-1.3.7.jar!/:na]
	at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:52) ~[resource-loader-1.3.7.jar!/:na]
	... 19 common frames omitted

I think using getResourceAsStream() as @LukeHornibrookHaventec propose could be a good alternative. And if you don't want to break existing behavior, we may introduce configuration to switch mode from (default) getResource() to getResourceAsStream().

Otherwise, may you open (move to public) method getSodiumPathInResources() that will help using SodiumJava(String absolutePath) constructor by letting client to manage copying native lib outside the fatjar.
Today I've to use reflection

File nativeLib = File.createTempFile("libsodium", null);
Method method = LibraryLoader.class.getDeclaredMethod("getSodiumPathInResources");
method.setAccessible(true);
String path = (String) method.invoke(null);

InputStream is = this.getClass().getClassloader().getResourceAsStream(path);
Files.copy(is, nativeLib.toPath(), StandardCopyOption.REPLACE_EXISTING);

LazySodiumJava lazySodiumJava = new LazySodiumJava(new SodiumJava(nativeLib.getAbsolutePath()));
commented

I'm not able to re-open, may I've to re-create the issue?