IOException when loading library from resources
mhilbush opened this issue · comments
I was attempting to update the lazysodium version that I use in openHAB from 4.0.1 to 4.2.4. When running with the new version I get an IOException when trying to execute:
LazySodiumJava lazySodiumJava = new LazySodiumJava(new SodiumJava());
Any idea what might be causing this?
co.libly.resourceloader.ResourceLoaderException: Failed to load the bundled library from resources by relative path (linux64/libsodium.so)
at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:65) ~[bundleFile:?]
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadBundledLibrary(LibraryLoader.java:115) ~[bundleFile:?]
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadLibrary(LibraryLoader.java:84) ~[bundleFile:?]
at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:34) ~[bundleFile:?]
at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:23) ~[bundleFile:?]
at org.openhab.binding.doorbird.internal.handler.DoorbellHandler.initialize(DoorbellHandler.java:128) [bundleFile:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.eclipse.smarthome.core.internal.common.AbstractInvocationHandler.invokeDirect(AbstractInvocationHandler.java:152) [bundleFile:?]
at org.eclipse.smarthome.core.internal.common.Invocation.call(Invocation.java:52) [bundleFile:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: java.io.IOException: Failed to list contents of /linux64/libsodium.so
at co.libly.resourceloader.ResourceLoader.doCopyDirectory(ResourceLoader.java:242) ~[bundleFile:?]
at co.libly.resourceloader.ResourceLoader.copyDirectory(ResourceLoader.java:234) ~[bundleFile:?]
at co.libly.resourceloader.ResourceLoader.getFileFromFileSystem(ResourceLoader.java:129) ~[bundleFile:?]
at co.libly.resourceloader.ResourceLoader.copyToTempDirectory(ResourceLoader.java:83) ~[bundleFile:?]
at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:52) ~[bundleFile:?]
... 15 more
The platform is Ubuntu 16.04
Good to hear from you again @mhilbush!
I just spun up a Ubuntu 16.04 VM and ran https://github.com/terl/lazysodium-examples which were all a success.
Could you git clone https://github.com/terl/lazysodium-examples and then cd into the java directory and do something like ./gradlew run --args 1 or ./gradlew run --args 2. If that runs without exceptions, then there's something else going on with your machine which we need to debug further.
I have an inkling that it might be that an incorrect GLIBC on your machine might be causing it to crash.
Both of these run successfully.
./gradlew run --args 1
./gradlew run --args 2
Some details on the box having the issue...
Linux openhab-md 4.15.0-66-generic #75~16.04.1-Ubuntu SMP Tue Oct 1 14:01:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
ldd (Ubuntu GLIBC 2.23-0ubuntu11) 2.23
And, btw, it works on a box running 18.04.
🤔 If ./gradlew run --args 1 ran successfully then that implies there's something wrong in the project you're instantiating LazySodium in. Are you using a JAR or are you using gradle to bring it in as a dependency?
Does your failing project's build.gradle file have the same versions numbers in https://github.com/terl/lazysodium-examples/blob/master/java/build.gradle#L23?
We use maven as the build system for openHAB. My dependencies in the pom.xml look like this.
<properties>
<dep.noembedding>jna</dep.noembedding>
</properties>
<dependencies>
<dependency>
<groupId>com.goterl.lazycode</groupId>
<artifactId>lazysodium-java</artifactId>
<version>4.2.4</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna</artifactId>
<version>5.4.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>co.libly</groupId>
<artifactId>resource-loader</artifactId>
<version>1.3.5</version>
<scope>compile</scope>
</dependency>
</dependencies>
This builds my jar to include all the classes and resources from the above dependencies. This is the same process as with lazysodium-java 4.0.1.
<properties>
<dep.noembedding>jna</dep.noembedding>
</properties>
<dependencies>
<dependency>
<groupId>com.goterl.lazycode</groupId>
<artifactId>lazysodium-java</artifactId>
<version>4.0.1</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>net.java.dev.jna</groupId>
<artifactId>jna</artifactId>
<version>5.4.0</version>
<scope>compile</scope>
</dependency>
</dependencies>
So, when I build my jar with the first set of deps, when it runs on my Ubuntu 16.04 platform, it fails with the error shown in my first post. When it runs on 18.04, it runs fine.
When I build with the second set of deps, it works fine on both platforms.
One thing I should point out... In both cases I'm specifying JNA 5.4.0, since that's the JNA version that openHAB includes. That's different than the JNA version specified by lazysodium 4.2.4.
Hang on. I can't seem to reproduce the problem at the moment. I'm having some type of other build issue. I'll update this when I can reproduce the problem again.
@gurpreet- I can reproduce the problem again. I've updated the above post to show exactly what I'm specifying for dependencies in my pom.xml.
Does Maven require you to specify dependencies of dependencies? Could you try removing the following or could you try upgrading it to 1.3.6?
<dependency>
<groupId>co.libly</groupId>
<artifactId>resource-loader</artifactId>
<version>1.3.5</version>
<scope>compile</scope>
</dependency>
I built with 1.3.6 (confirmed by examining the contents of my jar against the contents of the resource-loader jar downloaded from https://mvnrepository.com/artifact/co.libly/resource-loader/1.3.6). I get the same exception.
I see the exception is complaining about an absolute path name. Is that just what it says in the message, or does it really think that directory is off the filesystem root directory.
Caused by: java.io.IOException: Failed to list contents of /linux64/libsodium.so
Hi @mhilbush got busy with work, sorry.
Yes according to the stacktrace it believes that there is actually a file located in /linux64/libsodium.so.
It's failing this check jarUrl.toString().endsWith(".jar") or it could be throwing an exception anywhere in getThisJarPath. If any of those two things happen it falls back to getFileFromFileSystem, which just gets the .so files from the filesystem.
The weird thing is that your Ubuntu 16.04 is not correctly creating a temporary directory. The stacktrace should say something like:
/tmp/resource-loader/linux64/libsodium.so
This indicates that resource-loader cannot create a suitable temp directory. Does the process that runs Lazysodium have sufficient permissions to create files in that /tmp directory? Actually a better question might be what is the output of echo $TMPDIR on the 16.04 and the 18.04 and what permissions do the folders have?
resource-loader was my way of trying to solve the very difficult problem of both loading files out of JARs and also loading from the filesystem. The temporary directory tactic does not seem to be a cross-platform solution at all! I could shove everything in the home directory 🤔
Another useful variable is to inspect what Java thinks is the right place to store its temporary directories. Do a ps -ef | grep java then search the list for -Djava.io.tmpdir. If it's not set then it does need setting to a place where any program can read/write files.
got busy with work
No worries. I'm retired now, but I remember those days all too well. 😉
Does the process that runs Lazysodium have sufficient permissions to create files in that /tmp directory?
The process runs as me on both boxes. The permissions on /tmp are 777 on both boxes. Should no issue there.
what is the output of
echo $TMPDIR
The TMPDIR shell variable is not set on either box.
what Java thinks is the right place to store its temporary directories
My openHAB binding runs in a Karaf container.
On the 18.04 box:
-Djava.io.tmpdir=/opt/openhab2/userdata/tmp
drwxr-xr-x 9 mark mark 4096 Jan 30 07:18 /opt/openhab2/userdata/tmp
On the 16.04 box:
-Djava.io.tmpdir=/opt/openhab2/userdata/tmp
drwxr-xr-x 11 mark mark 4096 Jan 31 16:40 /opt/openhab2/userdata/tmp
Confirming the process is running as me:
mark 23198 2.7 13.9 8205608 2273292 ? Sl Jan30 54:46 /usr/bin/java -Dopenhab.home=/opt/openhab2 -Dopenhab.conf=/opt/openhab2/conf -Dopenhab.runtime=/opt/openhab2/runtime -Dopenhab.userdata=/opt/openhab2/userdata -Dopenhab.logdir=/opt/openhab2/userdata/logs -Dfelix.cm.dir=/opt/openhab2/userdata/config -Djava.library.path=/opt/openhab2/userdata/tmp/lib -Djetty.host=0.0.0.0 -Djetty.http.compliance=RFC2616 -Dorg.ops4j.pax.web.listening.addresses=0.0.0.0 -Dorg.osgi.service.http.port=8080 -Dorg.osgi.service.http.port.secure=8443 -Djava.awt.headless=true -XX:+UseG1GC -Djava.endorsed.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/jre/lib/endorsed:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed:/opt/openhab2/runtime/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/jre/lib/ext:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/opt/openhab2/runtime/lib/ext -Dkaraf.instances=/opt/openhab2/userdata/tmp/instances -Dkaraf.home=/opt/openhab2/runtime -Dkaraf.base=/opt/openhab2/userdata -Dkaraf.data=/opt/openhab2/userdata -Dkaraf.etc=/opt/openhab2/userdata/etc -Dkaraf.log=/opt/openhab2/userdata/logs -Dkaraf.restart.jvm.supported=true -Djava.io.tmpdir=/opt/openhab2/userdata/tmp -Djava.util.logging.config.file=/opt/openhab2/userdata/etc/java.util.logging.properties -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true -classpath /opt/openhab2/runtime/lib/boot/org.apache.karaf.diagnostic.boot-4.2.7.jar:/opt/openhab2/runtime/lib/boot/org.apache.karaf.jaas.boot-4.2.7.jar:/opt/openhab2/runtime/lib/boot/org.apache.karaf.main-4.2.7.jar:/opt/openhab2/runtime/lib/boot/org.apache.karaf.specs.activator-4.2.7.jar:/opt/openhab2/runtime/lib/boot/osgi.core-6.0.0.jar org.apache.karaf.main.Main```
Hmmm the plot indeed thickens. Can you try setting the -Djava.io.tmpdir=/opt/openhab2/userdata/tmp to a different directory? Also set the same directory for $TMPDIR
Can you try setting the -Djava.io.tmpdir=/opt/openhab2/userdata/tmp to a different directory?
I'm not sure that will be very easy to do. There are a lot of openHAB things that depend on this directory (i.e. there are hundreds of files located under /opt/openhab2/userdata/tmp.
Also set the same directory for $TMPDIR
This will be a lot easier. I'll give this a try first.
Thank you.
As it fails on this section:
final File[] srcFiles = srcDir.listFiles();
if (srcFiles == null) { // null if abstract pathname does not denote a directory, or if an I/O error occurs
throw new IOException("Failed to list contents of " + srcDir);
}
I found this interesting bug https://bugs.openjdk.java.net/browse/JDK-8179883 which states that listFiles only returns null if the program is ran on a command line 🤔
returns null if the program is ran on a command line
Hmm. Interesting.
- I'm running the same version of openjdk on both boxes.
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
- openHAB is started as a service at startup
I can shutdown the service and run openHAB from the command line to see if that makes a difference. I won't be able to do that tonight, so I'll give that a try tomorrow morning.
Here's another enlightening answer https://stackoverflow.com/a/55011870/3526705
The issue was the folder permissions. When running as a service, I don't have read access to the folder. When debugging I do. I'm using the same login credentials for both.
Wait, I think I misread that. Since it's not running from the command line, it shouldn't stumble across that error. Plus, it looks like that bug occurs on Windows.
Going back to something you said earlier...
It's failing this check jarUrl.toString().endsWith(".jar")
I wonder why would it be failing this test. The maven build process bundles the lazysodium and resource-loader classes into my openHAB binding jar file. I wonder if there could there be something about that that's causing a problem.
Does openHAB as a service have enough permissions? https://stackoverflow.com/a/55011870/3526705
It should. It explicitly sets the user & group to me.
[Unit]
Description=The openHAB 2 Home Automation Bus Solution
Documentation=http://docs.openhab.org
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=mark
Group=mark
GuessMainPID=yes
WorkingDirectory=/opt/openhab2
ExecStart=/opt/openhab2/start.sh server
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target
What happens if you set the 777 permissions on /opt/openhab2/userdata/tmp? Then restart the service and Java?
Going back to something you said earlier...
It's failing this check jarUrl.toString().endsWith(".jar")
I wonder why would it be failing this test. The maven build process bundles the
lazysodiumandresource-loaderclasses into my openHAB binding jar file. I wonder if there could there be something about that that's causing a problem.
I am sure that Lazysodium will still work out if it's in a JAR because those methods that verify if something is in a JAR or not are from Apache Commons. I do think this is a file/permission/location error. It works on 18.04, we just need to find out what's going on in 16.04 🤔
Changed perms of /opt/openhab2/userdata/tmp to 777, then restarted. Same result.
I don't know if this is good news or bad news. I have another openHAB installation running on 16.04. When I install my binding on that system, I also get the ResoureLoaderException.
So, at least we know it's not just something about that one box.
One other fact about that box... openHAB is running as root. So, from a permissions perspective, there should not be an issue with having insufficient permissions.
Adding another data point... I think I'm getting closer to the root cause of the issue.
The standard openHAB build process normally includes the contents (i.e. classes, resources, etc.) from the dependent libraries into the openHAB binding jar file. This is the process I've been using to do the build, and it has resulted in the above-described behavior. Here's a link to this jar file.
https://drive.google.com/open?id=1TyraDvlKldAUq2XKJnHkgpoCVA3Md5xz
As a test, I changed the build process to include the dependent jars into the openHAB binding jar file. My binding ran correctly using this build process. Here's a link to this jar file.
https://drive.google.com/open?id=1focoSOJ3ZFLwgh55rWo1jVhKMNPBCha3
So, it appears to have something to do with embedding the contents of the resource-loader and lazysodium-java jar files versus embedding the jar files themselves.
@gurpreet- I have an idea. Don't spend any more time on this until I can sort out a couple things.
Here's what I'm thinking... We can avoid the entire dependency embedding problem if you can make the lazysodium-java and resource-loader jars OSGI-compliant bundles. That way, the Apache Felix dependency resolver in openHAB can automatically download and install the jars from JCenter.
This may be accomplished as simply as adding the following to your build.gradle files.
apply plugin: 'osgi'
All this does, I think, is add some OSGI specific headers to the MANIFEST.MF.
When I have a little more to share, I post some additional information.
@mhilbush Great debugging and analysis. Dependencies of dependencies bugs are definitely hard to track.
I have no problem adding the osgi plugin if it comes down to it 👍
Update on progress so far.
Editing my post because I screwed something up.
I cloned both repos -- resource-loader and lazysodium-java. I added apply plugin 'osgi' to the build.gradle files in both repos and built the jars. I dropped the jars in openHAB's addons directory, and they were successfully recognized as OSGI bundles. Yay!
Then I changed the pom.xml for my openHAB binding to make the dependency scope provided instead of compile for resource-loader and lazysodium-java. I built my binding and dropped it in addons,
However, I get a NullPointerException on this line of code.
2020-02-02 16:59:06.280 [ERROR] [nal.common.AbstractInvocationHandler] - An error occurred while calling method 'ThingHandler.initialize()' on 'org.openhab.binding.doorbird.internal.handler.DoorbellHandler@1e4ec6ad': null
java.lang.NullPointerException: null
at co.libly.resourceloader.ResourceLoader.getFileFromFileSystem(ResourceLoader.java:120) ~[?:?]
at co.libly.resourceloader.ResourceLoader.copyToTempDirectory(ResourceLoader.java:83) ~[?:?]
at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:52) ~[?:?]
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadBundledLibrary(LibraryLoader.java:115) ~[?:?]
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadLibrary(LibraryLoader.java:88) ~[?:?]
at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:34) ~[?:?]
at org.openhab.binding.doorbird.internal.handler.DoorbellHandler.initialize(DoorbellHandler.java:130) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.eclipse.smarthome.core.internal.common.AbstractInvocationHandler.invokeDirect(AbstractInvocationHandler.java:152) [bundleFile:?]
at org.eclipse.smarthome.core.internal.common.Invocation.call(Invocation.java:52) [bundleFile:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
2020-02-02 16:59:06.287 [ERROR] [core.thing.internal.ThingManagerImpl] - Exception occurred while initializing handler of thing 'doorbird:d101:doorbell': null
java.lang.NullPointerException: null
at co.libly.resourceloader.ResourceLoader.getFileFromFileSystem(ResourceLoader.java:120) ~[?:?]
at co.libly.resourceloader.ResourceLoader.copyToTempDirectory(ResourceLoader.java:83) ~[?:?]
at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:52) ~[?:?]
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadBundledLibrary(LibraryLoader.java:115) ~[?:?]
at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadLibrary(LibraryLoader.java:88) ~[?:?]
at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:34) ~[?:?]
at org.openhab.binding.doorbird.internal.handler.DoorbellHandler.initialize(DoorbellHandler.java:130) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_232]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_232]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.eclipse.smarthome.core.internal.common.AbstractInvocationHandler.invokeDirect(AbstractInvocationHandler.java:152) [bundleFile:?]
at org.eclipse.smarthome.core.internal.common.Invocation.call(Invocation.java:52) [bundleFile:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Please disregard the initial version of my post above. I was on the wrong box when I ran the test the first time. I edited the post to reflect what happened when I ran the test on my 16.04 box.
Can you try replacing copyToTempDirectory with the following? I have added print statements, I want to see where it's failing...
public File copyToTempDirectory(String relativePath, Class outsideClass) throws IOException {
// If the file does not start with a separator,
// then let's make sure it does!
if (!relativePath.startsWith(File.separator)) {
relativePath = File.separator + relativePath;
}
// Create a "main" temporary directory in which
// everything can be thrown in.
File mainTempDir = createMainTempDirectory();
// Create the required directories.
mainTempDir.mkdirs();
try {
URL jarUrl = getThisJarPath(outsideClass);
System.out.println(jarUrl.getPath());
// Is the user loading this in a JAR?
if (jarUrl.toString().endsWith(".jar")) {
// If so the get the file/directory
// from a JAR
return getFileFromJar(jarUrl, mainTempDir, relativePath);
} else {
// If not then get the file/directory
// straight from the file system
System.out.println("Does not end in .jar");
return getFileFromFileSystem(relativePath, mainTempDir);
}
} catch (URISyntaxException e) {
// If we could not convert the jarUrl to a URI
// then it means we are not in a JAR,
// so we try load from the file system.
e.printStackTrace();
return getFileFromFileSystem(relativePath, mainTempDir);
}
}
/opt/openhab2/userdata/cache/org.eclipse.osgi/274/0/bundleFile
Does not end in .jar
So this is what Apache Felix does when it loads an OSGI compliant jar file into the runtime. That explains why it no longer ends in .jar.
Confirming the contents of the file.
Archive: /opt/openhab2/userdata/cache/org.eclipse.osgi/274/0/bundleFile
Length Date Time Name
--------- ---------- ----- ----
0 2020-02-02 09:50 META-INF/
1191 2020-02-02 09:50 META-INF/MANIFEST.MF
0 2020-02-02 07:21 com/
0 2020-02-02 07:21 com/goterl/
0 2020-02-02 07:21 com/goterl/lazycode/
0 2020-02-02 07:21 com/goterl/lazycode/lazysodium/
2307 2020-02-02 07:21 com/goterl/lazycode/lazysodium/SodiumJava.class
68919 2020-02-02 07:21 com/goterl/lazycode/lazysodium/LazySodium.class
1035 2020-02-02 07:21 com/goterl/lazycode/lazysodium/LazySodium$1.class
0 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/
1072 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/KeyPair.class
499 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/Constants.class
1187 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/DetachedDecrypt.class
662 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/Detached.class
616 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/LibraryLoadingException.class
742 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/DetachedEncrypt.class
3595 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/LibraryLoader.class
2315 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/Key.class
934 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/LibraryLoader$1.class
916 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/BaseChecker.class
1318 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/LibraryLoader$Mode.class
1064 2020-02-02 07:21 com/goterl/lazycode/lazysodium/utils/SessionPair.class
0 2020-02-02 07:21 com/goterl/lazycode/lazysodium/exceptions/
708 2020-02-02 07:21 com/goterl/lazycode/lazysodium/exceptions/SodiumException.class
10064 2020-02-02 07:21 com/goterl/lazycode/lazysodium/Sodium.class
0 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/
2092 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/PwHash.class
1118 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretStream.class
2438 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/PwHash$Alg.class
656 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash.class
1101 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretStream$Native.class
1136 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/StreamJava$Lazy.class
962 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$StateHMAC512256.class
594 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$State512$ByReference.class
551 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyExchange.class
899 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretBox$Lazy.class
337 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/DiffieHellman$Native.class
964 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretStream$State.class
1335 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/StreamJava$Method.class
953 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$StateHMAC256.class
1889 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$Native.class
299 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Random.class
1271 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/AEAD$Lazy.class
333 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Helpers$Lazy.class
266 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Helpers$Native.class
547 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/ShortHash.class
537 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/DiffieHellman.class
304 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Padding$Native.class
610 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Sign$Native.class
599 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Scrypt$Lazy.class
627 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretStream$Checker.class
1077 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Stream.class
1377 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/AEAD$Method.class
324 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/ShortHash$Native.class
2442 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$Lazy.class
720 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Box$Native.class
1310 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Scrypt.class
1358 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Stream$Method.class
314 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Helpers.class
1148 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Sign$Lazy.class
1028 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth.class
594 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/AEAD$StateAES$ByReference.class
921 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyDerivation$Checker.class
462 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$Checker.class
926 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$State256.class
1401 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Scrypt$Checker.class
462 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Sign$Checker.class
1803 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/PwHash$Checker.class
453 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretBox$Native.class
613 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecureMemory$Native.class
944 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$State512.class
1035 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$Native.class
969 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Box.class
250 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecureMemory$Lazy.class
338 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyDerivation$Native.class
924 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/StreamJava.class
554 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyDerivation$Lazy.class
462 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$Checker.class
771 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/StreamJava$Native.class
594 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$State256$ByReference.class
917 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyExchange$Lazy.class
550 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/GenericHash$Native.class
872 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/GenericHash$Lazy.class
515 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/ShortHash$Lazy.class
499 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Base.class
1237 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Hash$Lazy.class
1092 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretStream$Lazy.class
1082 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/AEAD.class
759 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/PwHash$Lazy.class
314 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Padding.class
417 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyExchange$Native.class
622 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretStream$State$ByReference.class
235 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Padding$Lazy.class
981 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Stream$Lazy.class
599 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/PwHash$Native.class
531 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/DiffieHellman$Lazy.class
848 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/AEAD$StateAES.class
1056 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Box$Checker.class
953 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$StateHMAC512.class
1265 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Box$Lazy.class
909 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Sign.class
574 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecretBox.class
1266 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Auth$Type.class
660 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/KeyDerivation.class
816 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Stream$Native.class
547 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/Scrypt$Native.class
334 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/SecureMemory.class
1325 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/AEAD$Native.class
796 2020-02-02 07:21 com/goterl/lazycode/lazysodium/interfaces/GenericHash.class
9599 2020-02-02 07:21 com/goterl/lazycode/lazysodium/LazySodiumJava.class
0 2020-02-02 07:21 armv6/
1196556 2020-02-02 07:21 armv6/libsodium.so
0 2020-02-02 07:21 mac/
773300 2020-02-02 07:21 mac/libsodium.dylib
0 2020-02-02 07:21 linux/
1121186 2020-02-02 07:21 linux/libsodium.so
0 2020-02-02 07:21 windows/
318976 2020-02-02 07:21 windows/libsodium.dll
0 2020-02-02 07:21 windows64/
270848 2020-02-02 07:21 windows64/libsodium.dll
0 2020-02-02 07:21 linux64/
1892087 2020-02-02 07:21 linux64/libsodium.so
--------- -------
5752259 122 files
I put in the following quick hack. It works.
if (jarUrl.toString().endsWith(".jar") || jarUrl.toString().endsWith("bundleFile")) {
Summarizing the changes I made to resource-loader and lazysodium-java. With respect to the test for /bundleFile, I don't know if there's a better way to check if it's an OSGI bundle. Have you tried using the ZipFile or JarFile classes to see if it's a jar file?
lazysodium-java
diff --git a/build.gradle b/build.gradle
index 988038c..513b756 100644
--- a/build.gradle
+++ b/build.gradle
@@ -16,6 +16,8 @@ plugins {
id "com.jfrog.bintray" version "1.8.4"
}
+apply plugin: 'osgi'
+
ext {
artifactId = "lazysodium-java"
groupId = "com.goterl.lazycode"
resource-loader
diff --git a/build.gradle b/build.gradle
index 77841ec..1be54e8 100644
--- a/build.gradle
+++ b/build.gradle
@@ -17,6 +17,8 @@ plugins {
id "com.jfrog.bintray" version "1.8.4"
}
+apply plugin: 'osgi'
+
ext {
artifactId = "resource-loader"
groupId = "co.libly"
@@ -205,4 +207,4 @@ tasks.withType(Test) {
}
}
}
-}
\ No newline at end of file
+}
diff --git a/src/main/java/co/libly/resourceloader/ResourceLoader.java b/src/main/java/co/libly/resourceloader/ResourceLoader.java
index 142b5e9..cf85f96 100644
--- a/src/main/java/co/libly/resourceloader/ResourceLoader.java
+++ b/src/main/java/co/libly/resourceloader/ResourceLoader.java
@@ -73,7 +73,7 @@ public class ResourceLoader {
try {
URL jarUrl = getThisJarPath(outsideClass);
// Is the user loading this in a JAR?
- if (jarUrl.toString().endsWith(".jar")) {
+ if (jarUrl.toString().endsWith(".jar") || jarUrl.toString().endsWith("/bundleFile")) {
// If so the get the file/directory
// from a JAR
return getFileFromJar(jarUrl, mainTempDir, relativePath);
Wow excellent find @mhilbush!
Yes adding a check for bundleFile would be a very quick fix. Do all OSGI containers end in bundleFile then once loaded into the runtime?
Hmm your question about seeing if it is a zip is a good one. Honestly, not sure if it will work but I'll try. Thank you for the diff of what you've done, it's super helpful 😀
Do all OSGI containers end in bundleFile then once loaded into the runtime?
I know it's the case for Apache Felix (one of several OSGI containers, and the only one I care about 😉), but I'm not sure about the others (e.g. Equinox).
Hmm your question about seeing if it is a zip is a good one. Honestly, not sure if it will work but I'll try.
This is the class I was looking at. I can try it in my local copy of resource-loader to see how it works in the OSGI world.
https://docs.oracle.com/javase/8/docs/api/java/util/jar/JarFile.html
I added this method to ResourceLoader.java.
private boolean isJarFile(URL jarUrl) {
if (jarUrl != null) {
try (JarFile jarFile = new JarFile(jarUrl.getPath())) {
// Successfully opened the jar file. Check if there's a manifest
// This is probably not necessary
Manifest manifest = jarFile.getManifest();
if (manifest != null) {
return true;
}
} catch (IOException | SecurityException | IllegalStateException e) {
System.out.println("Exception getting JarFile object: " + e.getMessage());
}
}
return false;
}
Then replaced this
if (jarUrl.toString().endsWith(".jar") || jarUrl.toString().endsWith("/bundleFile")) {
with this
if (isJarFile(jarUrl)) {
Worked like a charm.
Trialling it now on all platforms 🙂
Let's hope it works
This has been fixed in release 4.2.5. Test it out 😄
Looks good so far. Thanks!
-
I didn't see the new versions listed on Maven Central, but when ran the build for my binding, it pulled down the latest versions (4.2.5 and 1.3.7) as part of the build.
-
the lazysodium-java and resource-loader jars (aka OSGI bundles) were loaded just fine by the OSGI container in openHAB (thanks for making the jars OSGI-compliant!)
304 │ Active │ 80 │ 5.5.0 │ com.sun.jna
312 │ Active │ 80 │ 1.3.7 │ co.libly.resource-loader
313 │ Active │ 80 │ 4.2.5 │ com.goterl.lazycode.lazysodium-java
314 │ Active │ 80 │ 2.5.2.202002062320 │ org.openhab.binding.doorbird
- And, best of all, the libsodium library was loaded successfully on the 18.04 and 16.04 systems.
This is amazing news 🙌
Glad it has all got sorted. Thanks for your help once again!
I'll do some additional testing over the next couple days. If things continue to look good, I'll close this issue.
Thanks for all your help getting this sorted out.
@gurpreet- This is still looking good. I'm planning to include the new library versions in my next update.
But, I have one lingering question. In the current version of my code, I first try to load the bundled version, then, if that fails, try to load the OS version. Current code is here.
The current lazysodium-java version now uses the LibraryLoader.Mode enum to determine the loading order. By default, it looks like it tries to load the system library first, then tries the bundled version if that fails. If I want to maintain my current functionality, do I need to call SodiumJava(LibraryLoader.Mode.BUNDLED_ONLY then call SodiumJava(LibraryLoader.Mode.SYSTEM_ONLY if the bundled load fails?
@gurpreet- Did you have a chance to look at my question above?
Whoops so sorry, Github didn't notify me of it!
Yes your current implementation is correct. Have to try-catch it. I should add a "PREFER_BUNDLED" option 🤔
My openHAB changes were merged and all looks good. Thanks again for your help on this! Stay safe!!