[2.x] Autotools build can detect the wrong libjpeg
smcv opened this issue · comments
To reproduce
$ podman pull registry.gitlab.steamos.cloud/steamrt/scout/sdk:beta
$ podman run --rm -it registry.gitlab.steamos.cloud/steamrt/scout/sdk:beta
# cd
# git clone -bSDL2 https://github.com/libsdl-org/SDL_image
# cd SDL_image
(optionally apply #432 to fix #429)
# autoreconf -fi
# ./configure --enable-tests --disable-stb-image
(AVIF and JXL libraries are not found, this is OK)
# make
# make check VERBOSE=1
Note that in the scout SDK, /usr/include/jpeglib.h and /usr/lib/x86_64-linux-gnu/libjpeg.so are provided by libjpeg-turbo8-dev (libjpeg-turbo built to be compatible with the ABI of IJG libjpeg 8), but /usr/lib/x86_64-linux-gnu/ contains both:
/usr/lib/x86_64-linux-gnu/libjpeg.so -> libjpeg.so.8.0.2
/usr/lib/x86_64-linux-gnu/libjpeg.so.8 -> libjpeg.so.8.0.2
and
/usr/lib/x86_64-linux-gnu/libjpeg.so.62 -> libjpeg.so.62.0.0
where the latter is provided by IJG libjpeg version 6b.
Expected result
All tests pass.
Actual result
ERROR: 03/11/24 17:37:23: Assert 'Load .../test/sample.jpg (JPEG loading error)': Failed
A CMake build from the same sources succeeds.
Looking at the build log, one thing that jumps out at me is:
-- dynamic libjpeg -> libjpeg.so.62
This means that the libjpeg whose headers we #include (libjpeg-turbo, SONAME libjpeg.so.8) is not the same as the libjpeg that we dlopen (IJG libjpeg, SONAME libjpeg.so.62). libjpeg is sufficiently fault-tolerant that it manages to detect this incompatibility and report a recoverable error, rather than just crashing.
I think this is the root cause for why I experienced #428 and #429, which can only happen in an error-handling code path.
Workarounds
- Compile SDL_image on a system that only has one libjpeg. Production versions of the Steam Runtime are built on an Open Build Service instance in a minimal chroot, which does not exhibit this bug; but development builds are done in the non-minimal SDK environment, which does.
- Or, build with CMake. Some distros like Arch have switched to building the SDL family with CMake, although as far as I know, the SDL team still recommends Autotools for building SDL 2.x.
- Or, build with
--disable-jpg-sharedso that the chosen flavour of libjpeg is a hard dependency. We do this in Debian, Ubuntu, and the container-based Steam Runtime ≥ 2 branches; but we do not currently do this in Steam Runtime 1, so that the Steam Runtime 1 binary of SDL_image can be copied and pasted into games' portable non-Steam binary builds. - Or, build without
--disable-stb-imageso that stb_image is used instead of libjpeg (untested, but should work).
I tried to simulate this by putting libjpeg-turbo and ijg libraries in the same place, like:
$ ls -l ~/x2/lib/*
total 1200
lrwxrwxrwx. 1 ozkan ozkan 16 Mar 11 21:26 libjpeg.so -> libjpeg.so.9.6.0
lrwxrwxrwx. 1 ozkan ozkan 17 Mar 11 21:28 libjpeg.so.62 -> libjpeg.so.62.0.0
-rwxr-xr-x. 1 ozkan ozkan 291256 Dec 10 2013 libjpeg.so.62.0.0
lrwxrwxrwx. 1 ozkan ozkan 16 Mar 11 21:26 libjpeg.so.9 -> libjpeg.so.9.6.0
-rwxr-xr-x. 1 ozkan ozkan 929643 Mar 11 21:26 libjpeg.so.9.6.0
drwxrwxr-x. 2 ozkan ozkan 4096 Mar 11 21:26 pkgconfig.. and then, configuring SDL_image like LDFLAGS=-L~/x2/lib ../configure --disable-stb-image
gives me -- dynamic libjpeg -> libjpeg.so.62
When both are present:
- Which one do the headers actually belong?
- Which one does configure pick for you? and ..
- which one do you really want?
Linking a project with -ljpeg will select libjpeg.so -> libjpeg.so.9.6.0. So the headers (should) belong to 9.6.0.
CMake selects the correct library by following the symbolic links and then looking in the same directory for other files linking to libjpeg.so.9.6.0, of which it selects libjpeg.so.9.
libjpeg.so -> libjpeg.so.9.6.0 -> libjpeg.so.9.
- [To] Which one do the headers actually belong?
In my reproducer, libjpeg.so.8.
- Which one does configure pick for you?
libjpeg.so.62 (I think because 62 is more than 8, even though actually it's a strange way of writing 6b and is semantically more like 6.2)
- which one do you really want?
Whichever one the headers and the libjpeg.so symlink belong to, which in my case is libjpeg.so.8.
That's a fundamental deficiency of find_lib. Do we not have a way of following lib*.so symlinks by common shell tools?
For my use-case, it would be enough to have a configure option or an AC_ARG_VAR that lets me say "don't auto-detect, I'll tell you the right answer: it is libjpeg.so.8".
For my use-case, it would be enough to have a configure option or an
AC_ARG_VARthat lets me say "don't auto-detect, I'll tell you the right answer: it islibjpeg.so.8".
That actually sounds like a good solution to me
Do we not have a way of following
lib*.sosymlinks by common shell tools?
Messing with readlink result might help with this?
Messing with
readlinkresult might help with this?
It might be better not to make this critical-path for SDL 2.8.x.
Let's go with your AC_ARG_WITH solution then (but we should make it for every lib-to-be-detected in there.) Do you have a patch?
#434 does this for libjpeg, by overloading --enable-jpg-shared to take three classes of value:
- literal
no: link libjpeg in the normal way (not using dlopen), as is done in e.g. Debian - literal
yes: auto-detect what to dlopen - anything else: dlopen that SONAME
In practice, I think libjpeg is the only one of the libraries we link that has multiple competing implementations, and also the only one where the highest number is not necessarily the "best".
We could extend the same treatment to avif, jxl, png, tif relatively easily.
We cannot easily do the same thing for webp, because webp dlopens two libraries, not just one - so we'd have to invent a new --enable-webpdemux-shared=SONAME or something. I'd prefer not to get into that for 2.x, and certainly not for 2.8.x.
For 3.x, Autotools has gone away completely, which is part of why I don't think it's necessarily worth investing a whole lot of effort in it.