32-bit x86 build failure with 1.5.6

Question

32-bit x86 build failure with 1.5.6

ms178 opened this issue 2 months ago · comments

Describe the bug
Zstd 1.5.6 shows the following build failure on the 32-bit x86 version with both GCC 13.2.1 and LLVM/Clang-19 (shown below). As Zstd 1.5.5 builds just fine, I suspect a regression in 1.5.6.

[66/74] Linking C shared library lib/libzstd.so.1.5.6
FAILED: lib/libzstd.so.1.5.6 
: && /home/marcus/llvm19/bin/clang  -m32 -fPIC -O3 -march=native -mtune=native -mllvm -inline-threshold=1500 -mllvm -extra-vectorizer-passes -mllvm -enable-cond-stores-vec -mllvm -slp-vectorize-hor-store -mllvm -enable-loopinterchange -mllvm -enable-loop-distribute -mllvm -enable-unroll-and-jam -mllvm -enable-loop-flatten -mllvm -unroll-runtime-multi-exit -mllvm -aggressive-ext-opt -mllvm -enable-interleaved-mem-accesses -mllvm -enable-masked-interleaved-mem-accesses -fno-math-errno -fno-trapping-math -falign-functions=32 -funroll-loops -fno-semantic-interposition -fcf-protection=none -mharden-sls=none -fomit-frame-pointer -mprefer-vector-width=256 -flto -fwhole-program-vtables -fsplit-lto-unit -mllvm -adce-remove-loops -mllvm -enable-ext-tsp-block-placement -mllvm -enable-gvn-hoist -mllvm -enable-dfa-jump-thread -Wno-error -fprofile-use=/home/marcus/Downloads/zstd.profdata -fgnuc-version=6.5.0 -fexcess-precision=fast -freciprocal-math -fcx-limited-range -fdata-sections -ffunction-sections -fno-unique-section-names -fsplit-machine-functions -mtls-dialect=gnu2 -w -Wall -Wextra -Wundef -Wshadow -Wcast-align -Wcast-qual -Wstrict-prototypes -Qunused-arguments -Wa,--noexecstack -O3 -DNDEBUG  -Wl,--lto-CGO3 -Wl,--gc-sections -Wl,--icf=all -Wl,--lto-O3,-O3,-Bsymbolic-functions,--as-needed -fcf-protection=none -mharden-sls=none -Wl,-mllvm -Wl,-extra-vectorizer-passes -Wl,-mllvm -Wl,-enable-cond-stores-vec -Wl,-mllvm -Wl,-slp-vectorize-hor-store -Wl,-mllvm -Wl,-enable-loopinterchange -Wl,-mllvm -Wl,-enable-loop-distribute -Wl,-mllvm -Wl,-enable-unroll-and-jam -Wl,-mllvm -Wl,-enable-loop-flatten -Wl,-mllvm -Wl,-unroll-runtime-multi-exit -Wl,-mllvm -Wl,-aggressive-ext-opt -Wl,-mllvm -Wl,-enable-interleaved-mem-accesses -Wl,-mllvm -Wl,-enable-masked-interleaved-mem-accesses -march=native -maes -mbmi2 -mpclmul -flto -fwhole-program-vtables -fuse-ld=lld -Wl,-zmax-page-size=0x200000 -fprofile-use=/home/marcus/Downloads/zstd.profdata -Wl,-mllvm -Wl,-adce-remove-loops -Wl,-mllvm -Wl,-enable-ext-tsp-block-placement -Wl,-mllvm -Wl,-enable-gvn-hoist -Wl,-mllvm -Wl,-enable-dfa-jump-thread -Wl,--push-state -Wl,-whole-archive -ljemalloc_pic -Wl,--pop-state -lpthread -lstdc++ -lm -ldl -Wl,-z,now -Wl,-z,relro -Wl,-z,pack-relative-relocs -Wl,--hash-style=gnu -Wl,--undefined-version -shared -Wl,-soname,libzstd.so.1 -o lib/libzstd.so.1.5.6 lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/debug.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/entropy_common.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/error_private.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/fse_decompress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/pool.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/threading.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/xxhash.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/common/zstd_common.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/fse_compress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/hist.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/huf_compress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_compress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_compress_literals.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_compress_sequences.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_compress_superblock.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_double_fast.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_fast.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_lazy.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_ldm.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstd_opt.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/compress/zstdmt_compress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/decompress/huf_decompress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/decompress/zstd_ddict.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/decompress/zstd_decompress.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/decompress/zstd_decompress_block.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/decompress/huf_decompress_amd64.S.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/dictBuilder/cover.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/dictBuilder/divsufsort.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/dictBuilder/fastcover.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/dictBuilder/zdict.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v01.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v02.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v03.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v04.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v05.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v06.c.o lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/legacy/zstd_v07.c.o   && :
ld.lld: error: lib/CMakeFiles/libzstd_shared.dir/tmp/makepkg/lib32-zstd/src/zstd-1.5.6/lib/decompress/huf_decompress_amd64.S.o is incompatible with elf32-i386

Desktop (please complete the following information):

OS: CachyOS
Version: Rolling Release
Compiler: GCC 13.2.1 and Clang 19-git
Flags: see above
Other relevant hardware specs: Intel Haswell E5-2696 V3
Build system: See
PKGBUILD.txt for further build details.

Nick Terrell · Answer 1 · Tue Apr 02 2024 00:50:07 GMT+0800 (China Standard Time)

Looks like the repro command, taken from the PKGBUILD.txt file is:

export CC="clang -m32"
export CXX="clang++ -m32"

cmake -S build/cmake -B build -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_STANDARD=23 \
    -DCMAKE_CXX_STANDARD=23 \
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DCMAKE_INSTALL_LIBDIR=/usr/lib32 \
    -DZSTD_ZLIB_SUPPORT=ON \
    -DZSTD_LZMA_SUPPORT=ON \
    -DZSTD_LZ4_SUPPORT=ON \
    -DZSTD_BUILD_CONTRIB=ON \
    -DZSTD_BUILD_STATIC=OFF \
    -DZSTD_BUILD_TESTS=OFF \
    -DZSTD_PROGRAMS_LINK_SHARED=ON
cmake --build build

Its strange because we have 32-bit builds in our CI that are working. I'm traveling and don't have easy access to an intel machine that can do a 32-bit build. But I'll see if someone else can repro.

Felix Handte · Answer 2 · Tue Apr 02 2024 01:44:40 GMT+0800 (China Standard Time)

Zstd includes an optimized assembly Huffman decoder on x86_64, so building zstd not only invokes a C compiler (which you are telling to build in 32-bit mode) but also invokes an assembler (which you aren't).

I can repro your issue with your invocation, and can confirm it's fixed if you instead:

export CFLAGS="-m32"
export CXXFLAGS="-m32"
export ASMFLAGS="-m32"

It's curious that you say this worked in 1.5.5, since the asm decoder has existed for a while and I'm not aware of anything having changed here between those releases.

ms178 · Answer 3 · Tue Apr 02 2024 06:35:16 GMT+0800 (China Standard Time)

Thanks, Felix! Indeed, that fixes the observed failure. It is still puzzling why I didn't hit the same issue on 1.5.5 earlier. CachyOS upstream doesn't seem to hit this either, according to their PKGBUILD at https://github.com/CachyOS/CachyOS-PKGBUILDS/blob/master/zstd/lib32-zstd/PKGBUILD, they do not specify export ASMFLAGS="-m32" there but use GCC to compile the package.