LibraryLoader.search() path not taken into consideration unless also in LD_LIBRARY_PATH
ja2142 opened this issue · comments
On linux, when loading library outside of normal library paths, the LD_LIBRARY_PATH has to be set - if it isn't, library fails to load.
Example:
Let's say there's a lib at ~/mylib/libmylib.so. Trying to load it with:
MyLib lib = LibraryLoader.create(MyLib.class).failImmediately()
.search("~/mylib")
.load("libmylib.so");
Fails with java.lang.UnsatisfiedLinkError: libmylib.so: cannot open shared object file: No such file or directory
.
When the path in which library resides is added to LD_LIBRARY_PATH
(by export LD_LIBRARY_PATH=~/mylib
), code above loads the library without any problems.
Expected behaviour:
I'd expect LibraryLoader.search(path)
to work, without me having to deal with environment variables. Maybe a fix would be as easy as just prepending path from LibraryLoader.search()
to LD_LIBRARY_PATH
?
Why:
I'm trying to get a project with native libs packaged in a .jar to work. It works by unpacking these native libs into some directory in users home, and then loading them with jnr-ffi. Again I'd hope that LibraryLoader.search(path).load(lib)
would just work, without having to fiddle with env variables.
May be related to #129 (paths added via search()
seem to have priority, as long as they are in LD_LIBRARY_PATH
- otherwise they don't seem to be taken into consideration at all)
I'm fairly sure this is an issue because of ~
which wouldn't be interpreted as you expect in Java.
I believe ~
expanding to the current user's home directory is a bashism or at least a shell behavior that isn't expected to be found everywhere including Java.
Solutions
- Try replacing
~
yourself with the system environment variable representing the user's home directory, this can be found usingSystem.getEnv()
, maybe there are libraries or utilites that help with this, though it shouldn't be more than a quick string replace that replaces~
if found in the beginning of the string with the user's home directory which you'd need to query and find - Use the absolute path ie
/home/user/mylib/
, relative paths also work so you can additionally do something like../libs/mylib/
if you place it there, I myself prefer this option
Hope this helps
Sorry for the confusion, but I used ~ as a placeholder for my home directory, in code and in commands I used absolute paths (/home/user/...
) everywhere. Although now that I tested it, it works exactly the same when I do use ~
(both in code and export LD_LIBRARY_PATH=...
).
I guess the problem might be with some dlopen, or whatever underlying mechanism is being used that only looks in predefined library paths, and LD_LIBRARY_PATH?
Alright, I've dug a little deeper, and I've discovered couple of things:
- Even though, as @basshelal said, it's possible to load a library with it's full name using a relative or absolute path like so:
LibraryLoader.create(Lib.class).load("/home/ja/mylib/libmylib.so");
It's not possible to use full name of library, when not providing a path, because in such case LibraryLoader assumes that short name is being used (i.e. load("mylib")
and load("mylib/libmylib.so")
are ok, but load("libmylib.so")
, or load("libmylib.so.3")
isn't). The solution to my original problem is just to use short name.
- My original problem was that libraries installed on my system were loaded before libs contained in a jar. I thought that this could be solved by just trying to load proper version of libraries - i.e.
load("libavutil.so.56")
- but, as I've learned this doesn't seem to be possible. In "GettingStarted.md" there's an example that usesload("c")
- and although this is probably fine for something as stable as libc, I'd say it's a bad idea for pretty much any other library - one should really load a specific major version, because different major version are very likely to not be binary compatible. Indeed, my original problem was caused by system installedlibavutil.so.57
being load instead oflibavutil.so.56
contained in a jar, which resulted in a crash. Clearly, the strategy to load latest version of a library (which seems to be what jnr is doing) is not a very good one. Sure I could just load from specific path, but I'd say that I don't really care from where library is loaded, as long as it's compatible (which for most libraries should be the case as long as major version matches). I think a good solution would be to just allow loading specific version of lib, without forcing only one path to be searched (so in essence allow what I did in my original question) - it looks like this is already possible on windows.
For now I guess there are two workarounds that more or less work for me:
- using path in load:
LibraryLoader.create(MyLib.class).failImmediately()
.load("/home/user/mylib/libmylib.so");
- or adding
PreferCustomPaths
option:
LibraryLoader.create(MyLib.class).failImmediately()
.option(LibraryOption.PreferCustomPaths, true)
.search("~/mylib")
.load("mylib");