lovell / sharp-libvips

Hi! Thanks so much for the work put into this project, having a static build for libvips is incredibly useful!

I've just had one problem with it; I'm binding to the C api but certain function symbols don't seem to be visible in the final output, eg. vips_pngsave*. Compared to jpegsave which definitely is visible:

$ nm -gD lib/libvips-cpp.so.42 | grep vips_jpegsave
0000000000141140 T vips_jpegsave
00000000001412a0 T vips_jpegsave_buffer
0000000000141390 T vips_jpegsave_mime
00000000001411f0 T vips_jpegsave_target

$ nm -gD lib/libvips-cpp.so.42 | grep vips_pngsave
(no results)

(of course the CPP api for pngsave is visible:)

$ nm -gDC lib/libvips-cpp.so.42 | grep pngsave
0000000000133550 T vips::VImage::pngsave_buffer(vips::VOption*) const
0000000000133600 T vips::VImage::pngsave_target(vips::VTarget, vips::VOption*) const
00000000001334a0 T vips::VImage::pngsave(char const*, vips::VOption*) const

(this is on libvips-8.14.2-linux-x64, but I get the same result on all the linux- and darwin- variants I've tried)

I've not got much experience with C or C++ development, and definitely don't have much experience with compiler/linker behaviour, so I've not been able to figure out why the png symbols are getting dropped. I have been able to work around the problem by building with libvips (not -cpp) as my target - ie. andrew-nowak@d7802f7 which is perfect for my needs but of course not suitable here.

I know this project is primarily to support https://github.com/lovell/sharp which uses the c++ api, so possibly this issue is out of scope?

Does removing the -Wl,-Bsymbolic-functions linker flag help, as that can prevent functions from being exported?

sharp-libvips/build/lin.sh

Line 455 in b77f938

    
           ${LINUX:+-Dcpp_link_args="$LDFLAGS -Wl,-Bsymbolic-functions -Wl,--version-script=$DEPS/vips/vips.map $EXCLUDE_LIBS"}

unfortunately it doesn't seem to; I tried building with

-  ${LINUX:+-Dcpp_link_args="$LDFLAGS -Wl,-Bsymbolic-functions -Wl,--version-script=$DEPS/vips/vips.map $EXCLUDE_LIBS"}
+  ${LINUX:+-Dcpp_link_args="$LDFLAGS -Wl,--version-script=$DEPS/vips/vips.map $EXCLUDE_LIBS"}

but I still see no vips_pngsave

Thanks for checking. This appears to relate to libvips supporting either libpng or libspng, with a preference for the latter. I tried removing libspng from the build, so libpng was detected and used, and the pngsave* symbols were exported as expected.

-mkdir ${DEPS}/spng
-$CURL https://github.com/randy408/libspng/archive/v${VERSION_SPNG}.tar.gz | tar xzC ${DEPS}/spng --strip-components=1
-cd ${DEPS}/spng
-CFLAGS="${CFLAGS} -O3 -DSPNG_SSE=4" meson setup _build --default-library=static --buildtype=release --strip --prefix=${TARGET} ${MESON} \
-  -Dstatic_zlib=true
-meson install -C _build --tag devel

$ nm -gDC lib/libvips-cpp.so.42 | grep pngsave
0000000000147840 T vips_pngsave
00000000001478f0 T vips_pngsave_buffer
00000000001479e0 T vips_pngsave_target
00000000001348b0 T vips::VImage::pngsave_buffer(vips::VOption*) const
0000000000134960 T vips::VImage::pngsave_target(vips::VTarget, vips::VOption*) const
0000000000134800 T vips::VImage::pngsave(char const*, vips::VOption*) const

My best guess would be that this has something to do with multiple compilation units and linker optimisations, but I'd need to dig into it a bit deeper to understand exactly what's happening.

I think I've worked out what's happening here.

The definition of vips_pngsave and friends is in pngsave.c, which also houses all the libpng-related logic. When libvips is compiled with libspng support, and therefore without libpng support, no other compilation unit references vips_pngsave (the C++ bindings invoke operations by name, skipping the C API).

This means that the dead code elimination flag -Wl,--gc-sections here results in their removal from the final statically-built shared library.

sharp-libvips/build/lin.sh

Line 52 in a8f9b80

export LDFLAGS+=" -Wl,--gc-sections -Wl,-rpath=\$ORIGIN/"

https://gcc.gnu.org/onlinedocs/gnat_ugn/Compilation-options.html

So you should be able to remove -Wl,--gc-sections to keep these otherwise-unused functions, albeit at a larger file size.

You might also be interested in https://github.com/kleisauke/libvips-packaging a fork that I maintain for NetVips that distributes libvips.so.42 (in addition to libvips-cpp.so.42) and therefore does not remove the definition of vips_pngsave and friends as part of DCE.

$ mkdir linux-x64
$ curl -fsSL https://github.com/kleisauke/libvips-packaging/releases/download/v8.14.2/libvips-8.14.2-linux-x64.tar.gz | tar xzC linux-x64
$ nm -gD linux-x64/lib/libvips.so.42 | grep vips_pngsave
0000000000127740 T vips_pngsave
00000000001277f0 T vips_pngsave_buffer
00000000001278e0 T vips_pngsave_target

@lovell Apologies for not responding sooner; thanks so much for the explanation, I have learnt several things from this thread!!

@kleisauke Thanks for the pointer! That sounds very much like what I need, I'll make sure to take a look.

I think my question has been answered and I've got a few ways forward, so thanks once again to both for your help!

Missing some symbols from the libvips C api