Android: remove versioning from soname

Question

Android: remove versioning from soname

pazos opened this issue 4 years ago · comments

So all libraries are called whatever.so instead of whatever.so.x.y.

This is a prerequisite for koreader/android-luajit-launcher#175 and would improve koreader/koreader#6881 by moving libraries extraction from first run to install time.

I have no idea how to do that, pinging @NiLuJe

NiLuJe · Answer 1 · Tue Nov 24 2020 04:47:49 GMT+0800 (China Standard Time)

Most of 'em already do that by default, otherwise it's messy, and, for autotools stuff, it involves reverting the libtool hack that we need for I don't even remember what that disabled Android-specific quirks, one of which was that soname trickery, IIRC.

As for CMake stuff, jpeg-turbo is broken (again, don't remember how) when built in Android mode, which also did the same thing to the soname, besides breaking it ;p.

NiLuJe · Answer 2 · Tue Nov 24 2020 04:48:51 GMT+0800 (China Standard Time)

Doing it post-build involves modifying the ELF header, which patchelf can probably do, but then patchelf is known to randomly break some binaries, so, eh ^^.

Frans de Jonge · Answer 3 · Tue Nov 24 2020 05:10:09 GMT+0800 (China Standard Time)

A prerequisite? Seems unlikely. ;-) For many of them you could simply remove my patch, but I put that in there to work around specific Android issues (among other things it's a simple hack to avoid clashes with system libs like zlib iirc 'cause bionic kinda sucks). Cf. #521

…

On Mon, Nov 23, 2020 at 9:49 PM NiLuJe ***@***.***> wrote: Doing it post-build involves modifying the ELF header, which patchelf can probably do, but then patchelf is known to randomly break some binaries, so, eh ^^. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABRQBJM5WOQ7UJSIWJ67ILSRLDEDANCNFSM4T76OKTA> .

Martín Fernández · Answer 4 · Tue Nov 24 2020 05:13:56 GMT+0800 (China Standard Time)

Uuuuuh, In principle adding -Wl,-soname whatever.so in ldflags is what the NDK does automatically. I see that we do that for some libs: https://github.com/koreader/koreader-base/search?q=-Wl%2C-soname

As I don't have any idea about autotools (or generally about build tools) I have no idea if this is feasible.

Anyways, the list of libs that need a change in soname are:

Martín Fernández · Answer 5 · Tue Nov 24 2020 05:19:41 GMT+0800 (China Standard Time)

For many of them you could simply remove my patch, but I put that in there
to work around specific Android issues (among other things it's a simple
hack to avoid clashes with system libs like zlib iirc 'cause bionic kinda
sucks). Cf. #521

Oh, that's interesting. Indeed they can clash with system libraries, but I have no way to check that. Did it happen at some point?

Frans de Jonge · Answer 6 · Tue Nov 24 2020 05:25:48 GMT+0800 (China Standard Time)

Yes, see some of the linked issues. That libjpeg.so.8 you mentioned is guaranteed to break. You can of course change the name to androidsuckslibjpeg.so instead.

…

On Mon, Nov 23, 2020 at 10:19 PM Martín Fernández ***@***.***> wrote: For many of them you could simply remove my patch, but I put that in there to work around specific Android issues (among other things it's a simple hack to avoid clashes with system libs like zlib iirc 'cause bionic kinda sucks). Cf. #521 <#521> Oh, that's interesting. Indeed they can clash with system libraries, but I have no way to check that. Did it happen at some point? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABRQBM6S5ICFM3U62N4BKLSRLGX3ANCNFSM4T76OKTA> .

NiLuJe · Answer 7 · Tue Nov 24 2020 05:42:40 GMT+0800 (China Standard Time)

FWIW, many (if not all) of those are indeed libtool-generated ;).

(Possible special exception for OpenSSL, which uses a truly horrendous custom buildsystem, but it might ultimately use libtool for those anyway).

Martín Fernández · Answer 8 · Wed Nov 25 2020 01:30:48 GMT+0800 (China Standard Time)

Yes, see some of the linked issues. That libjpeg.so.8 you mentioned is guaranteed to break.
You can of course change the name to androidsuckslibjpeg.so instead.

I will try to revert #521 and see what crashes before changing library names. Also I think part of https://github.com/koreader/android-luajit-launcher/blob/master/assets/dl.lua was meant to address the fact that we're loading libs from filesDir instead of libraryDir, but I never played with the code. Let's see.

Frans de Jonge · Answer 9 · Wed Nov 25 2020 05:05:51 GMT+0800 (China Standard Time)

I will try to revert #521 and see what crashes before changing library names.

On many of our devices (un)fortunately nothing. A few of them will crash on Android <=4.2 or 8+ guaranteed though.

Also I think part of https://github.com/koreader/android-luajit-launcher/blob/master/assets/dl.lua was meant to address the fact that we're loading libs from filesDir instead of libraryDir, but I never played with the code. Let's see.

Have fun fighting with Bionic. Especially on 4.x. ;-)

Martín Fernández · Answer 10 · Tue Dec 22 2020 04:29:44 GMT+0800 (China Standard Time)

I will try to revert #521 and see what crashes before changing library names.

On many of our devices (un)fortunately nothing. A few of them will crash on Android <=4.2 or 8+ guaranteed though.

Also I think part of https://github.com/koreader/android-luajit-launcher/blob/master/assets/dl.lua was meant to address the fact that we're loading libs from filesDir instead of libraryDir, but I never played with the code. Let's see.

Have fun fighting with Bionic. Especially on 4.x. ;-)

I have good news 😄

These kind of issues, like:

12-21 21:06:04.314  8277  8300 I KOReader: Android 11 - R (API 30) - flavor: rocks
12-21 21:06:04.366  8277  8300 I KOReader:  initializing for device sdk_gphone_x86_64_arm64
12-21 21:06:04.366  8277  8300 I KOReader:  framebuffer resolution: {
12-21 21:06:04.366  8277  8300 I KOReader:     ["w"] = 1080,
12-21 21:06:04.366  8277  8300 I KOReader:     ["h"] = 1920,
12-21 21:06:04.366  8277  8300 I KOReader: }
12-21 21:06:04.369  8277  8300 V dlopen  : dl.dlopen - library lname detected /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:06:04.370  8277  8300 V dlopen  :          dl.dlopen - opening needed libz.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:06:04.370  8277  8300 V dlopen  : dl.dlopen - library lname detected /system/lib/libz.so
12-21 21:06:04.371  8277  8300 V dlopen  : sys_dlopen - loading library /system/lib/libz.so
12-21 21:06:04.371  8277  8300 V dlopen  :          dl.dlopen - opening needed libjpeg.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:06:04.371  8277  8300 V dlopen  : dl.dlopen - library lname detected /system/lib/libjpeg.so
12-21 21:06:04.372  8277  8300 V dlopen  : sys_dlopen - loading library /system/lib/libjpeg.so
12-21 21:06:04.373  8277  8300 E KOReader: [NativeThread]: failed to run script: setupkoenv.lua:27: Not able to load dynamic library: libs/libmupdf.so
12-21 21:06:04.373  8277  8300 E KOReader: [NativeThread]: Stopping due to previous errors

Are reproducible everywhere if we have an unversioned libjpeg.so in android.filesDir
Are not reproducible when we bundle our own copy of libjpeg.so in android.nativeLibraryDir 👏👏👏

So the flow:

unversioning
move them to android.nativeLibraryDir
do symlinks where they're expected

Produces a very reliable APK, except for rapidjson.so, which is missing for some unrelated reason :p

Frans de Jonge · Answer 11 · Tue Dec 22 2020 04:44:03 GMT+0800 (China Standard Time)

That's great, even on 4.0? 💯

Martín Fernández · Answer 12 · Tue Dec 22 2020 04:50:04 GMT+0800 (China Standard Time)

I don't have an emulator for 14-15, and the emulator for 21 is broken. But works on the rest of them and actual devices

This is the full thing for mupdf with all libraries in place

12-21 21:33:38.955  8463  8514 I KOReader: Android 11 - R (API 30) - flavor: rocks
12-21 21:33:39.010  8463  8514 I KOReader:  initializing for device sdk_gphone_x86_64_arm64
12-21 21:33:39.010  8463  8514 I KOReader:  framebuffer resolution: {
12-21 21:33:39.010  8463  8514 I KOReader:     ["h"] = 1920,
12-21 21:33:39.010  8463  8514 I KOReader:     ["w"] = 1080,
12-21 21:33:39.010  8463  8514 I KOReader: }
12-21 21:33:39.014  8463  8514 V dlopen  : dl.dlopen - library lname detected /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.015  8463  8514 V dlopen  :          dl.dlopen - opening needed libz.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.015  8463  8514 V dlopen  : dl.dlopen - library lname detected /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libz.so
12-21 21:33:39.015  8463  8514 V dlopen  :          dl.dlopen - opening needed libc.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libz.so
12-21 21:33:39.015  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libc.so
12-21 21:33:39.016  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libc.so
12-21 21:33:39.016  8463  8514 V dlopen  : sys_dlopen - loading library /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libz.so
12-21 21:33:39.016  8463  8514 V dlopen  :          dl.dlopen - opening needed libjpeg.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.017  8463  8514 V dlopen  : dl.dlopen - library lname detected /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libjpeg.so
12-21 21:33:39.018  8463  8514 V dlopen  :          dl.dlopen - opening needed libc.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libjpeg.so
12-21 21:33:39.018  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libc.so
12-21 21:33:39.019  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libc.so
12-21 21:33:39.019  8463  8514 V dlopen  : sys_dlopen - loading library /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libjpeg.so
12-21 21:33:39.019  8463  8514 V dlopen  :          dl.dlopen - opening needed libfreetype.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.019  8463  8514 V dlopen  : dl.dlopen - library lname detected /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libfreetype.so
12-21 21:33:39.020  8463  8514 V dlopen  :          dl.dlopen - opening needed libc.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libfreetype.so
12-21 21:33:39.020  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libc.so
12-21 21:33:39.021  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libc.so
12-21 21:33:39.021  8463  8514 V dlopen  : sys_dlopen - loading library /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libfreetype.so
12-21 21:33:39.021  8463  8514 V dlopen  :          dl.dlopen - opening needed libharfbuzz.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.022  8463  8514 V dlopen  : dl.dlopen - library lname detected /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libharfbuzz.so
12-21 21:33:39.022  8463  8514 V dlopen  :          dl.dlopen - opening needed libm.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libharfbuzz.so
12-21 21:33:39.022  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libm.so
12-21 21:33:39.023  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libm.so
12-21 21:33:39.023  8463  8514 V dlopen  :          dl.dlopen - opening needed libfreetype.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libharfbuzz.so
12-21 21:33:39.023  8463  8514 V dlopen  : dl.dlopen - library lname detected /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libfreetype.so
12-21 21:33:39.024  8463  8514 V dlopen  :          dl.dlopen - opening needed libc.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libfreetype.so
12-21 21:33:39.024  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libc.so
12-21 21:33:39.024  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libc.so
12-21 21:33:39.024  8463  8514 V dlopen  : sys_dlopen - loading library /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libfreetype.so
12-21 21:33:39.025  8463  8514 V dlopen  :          dl.dlopen - opening needed libc.so for /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libharfbuzz.so
12-21 21:33:39.025  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libc.so
12-21 21:33:39.025  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libc.so
12-21 21:33:39.025  8463  8514 V dlopen  : sys_dlopen - loading library /data/app/~~FJZspMVmEGU3lQXB9zLVLA==/org.koreader.launcher-Q1j57HciDDhC2ZRPuXmjIg==/lib/x86/libharfbuzz.so
12-21 21:33:39.026  8463  8514 V dlopen  :          dl.dlopen - opening needed libm.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.026  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libm.so
12-21 21:33:39.026  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libm.so
12-21 21:33:39.027  8463  8514 V dlopen  :          dl.dlopen - opening needed liblog.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.027  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/liblog.so
12-21 21:33:39.028  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/liblog.so
12-21 21:33:39.028  8463  8514 V dlopen  :          dl.dlopen - opening needed libc.so for /data/user/0/org.koreader.launcher/files/libs/libmupdf.so
12-21 21:33:39.028  8463  8514 V dlopen  : dl.dlopen - library lname detected /system/lib/libc.so
12-21 21:33:39.028  8463  8514 V dlopen  : sys_dlopen - loading library /system/lib/libc.so