Cross-compiling result in ImportError: libffi.so.8: cannot open shared object file: No such file or directory

[Vincent Fortier]

While preparing an update for our python-3.8 package for Synology NAS (uses linux called DSM), the CI was able to cross-compile all packages and dependencies with the only exception, cross-compiling x86_64 (all builds are done from a x86_64 host docker image). My initial thoughts are that it somehow loose track of the library path, hence trying to find libffi.so from the host default path (but I may well be wrong).

PR is SynoCommunity/spksrc#4902 and last CI run is https://github.com/SynoCommunity/spksrc/pull/4902/checks?check_run_id=3858396685 where it failed only on x64 arch for DSM version 6.1 and 7.0. All packages are built using Synology toolchain for each associated architectures.

Using version 1.4.0 from https://pypi.org/project/PyNaCl/ or latest snapshot from github gets to the exact same error.

Cross-compiling using following versions:

• python: 3.8.12
• libffi: 3.4.2
• cffi (pip/wheel): 1.14.6, 1.14.5
• PyNaCl: 1.3.0, 1.4.0, github-master

End of the build error:

 make[4]: Leaving directory '/home/spksrc/python38-update-fix/spksrc/cross/libsodium'
===>  Extracting for PyNaCl
tar -xzpf /home/spksrc/python38-update-fix/spksrc/cross/PyNaCl/../../distrib/PyNaCl-gita5a7876b523056241b53a49b7e53e7de0d8e7ccd.tar.gz -C /home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0 
===>  Patching for PyNaCl
===>  Configuring for PyNaCl
===>  Compiling for PyNaCl
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 203, in <module>
    setup(
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/spksrc/python38-update-fix/spksrc/native/python38/work-native/install/usr/local/lib/python3.8/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/setuptools/dist.py", line 453, in __init__
    _Distribution.__init__(
  File "/home/spksrc/python38-update-fix/spksrc/native/python38/work-native/install/usr/local/lib/python3.8/distutils/dist.py", line 292, in __init__
    self.finalize_options()
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/setuptools/dist.py", line 831, in finalize_options
    ep(self)
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/setuptools/dist.py", line 852, in _finalize_setup_keywords
    ep.load()(self, ep.name, value)
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 219, in cffi_modules
    add_cffi_module(dist, cffi_module)
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
    execfile(build_file_name, mod_vars)
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 25, in execfile
    exec(code, glob, glob)
  File "src/bindings/build.py", line 35, in <module>
    ffi = FFI()
  File "/home/spksrc/python38-update-fix/spksrc/spk/homeassistant/work-x64-7.0/install/var/packages/homeassistant/target/lib/python3.8/site-packages/cffi/api.py", line 48, in __init__
    import _cffi_backend as backend
ImportError: libffi.so.8: cannot open shared object file: No such file or directory
make[3]: *** [../../mk/spksrc.python-wheel.mk:33: build_python_wheel] Error 1
make[3]: Leaving directory '/home/spksrc/python38-update-fix/spksrc/cross/PyNaCl'
make[2]: *** [../../mk/spksrc.depend.mk:51: depend_target] Error 2
make[2]: Leaving directory '/home/spksrc/python38-update-fix/spksrc/spk/homeassistant'
make[1]: Leaving directory '/home/spksrc/python38-update-fix/spksrc/spk/homeassistant'

While the libraries are available:

$ find . -name libffi.so.8
./work-x64-7.0/install/var/packages/homeassistant/target/lib/libffi.so.8
./work-x64-7.0/libffi-3.4.2/x86_64-pc-linux-gnu/.libs/libffi.so.8
./work-x64-6.1/libffi-3.4.2/x86_64-pc-linux-gnu/.libs/libffi.so.8
./work-x64-6.1/install/var/packages/homeassistant/target/lib/libffi.so.8

And the work-x64-<DSM-version>/install/var/packages/homeassistant/target/lib is part of the library path.

[Paul Kehrer]

This is a cross-compile error with cffi, which pyNaCl does depend on, but is not something we control.

[Vincent Fortier]

I was able to found the reason why: crossenv

Our build framework starts by building a minimal python38 and includes the necessary wheels, including crossenv.
It happens that all version >=1.0 of crossenv makes PyNaCl to fail to build in our cross-compiling environment, only for x86_64 architecture.

I presume that crossenv has a bug that gets triggered in cross-compiling environment where if the target is of the same arch of the host it goes outside of its boundary instead of using the passed library paths of our toolchain.