Cmake tries to install to `/usr/local/lib/python3.11/site-packages` instead of the local environment
jhossbach opened this issue · comments
Hey,
I am currently trying to make the espresso MD package pip-installable. I had successfully installed the package using scikit-build
with the following setup.py
:
from skbuild import setup
setup(
name="espressomd",
version="4.3",
description="Extensible Simulation Package for Research on Soft Matter Systems",
author="The ESPResSo project",
url="https://espressomd.org",
license_files=["COPYING"],
)
But now, when moving to scikit-build-core
, I ran into the following issue
scikit_build_core - INFO - RUN: /tmp/pip-build-env-tv8rqy58/overlay/lib/python3.11/site-packages/cmake/data/bin/cmake --install /tmp/tmpkzta0wja/build --prefix /tmp/tmpkzta0wja/wheel/platlib --strip
-- Install configuration: "Release"
CMake Error at /tmp/tmpkzta0wja/build/src/shapes/cmake_install.cmake:60 (file):
file cannot create directory:
/usr/local/lib/python3.11/site-packages/espressomd. Maybe need
administrative privileges.
This is the pyproject.toml
:
[build-system]
requires = [
"scikit-build-core",
"cmake>=3.18",
"numpy>=1.23",
"cython>=0.29.21,<=3.0.7",
]
build-backend = "scikit_build_core.build"
[project]
name = "espressomd"
version = "4.3"
This is the output of python -m scikit_build_core.builder
:
Scikit-build-core 0.8.2 on Python 3.11.8 (main, Feb 12 2024, 14:50:05) [GCC 13.2.1 20230801]
Detected Python Library: /usr/lib/libpython3.11.so
Detected ABI3 Python Library: /usr/lib/libpython3.11.so
Detected Python Include Directory: /usr/include/python3.11
Detected NumPy Include Directory: None
Detected Platform: linux-x86_64
Detected SOABI: cpython-311-x86_64-linux-gnu
Detected ABI3 SOABI: abi3
Default Wheel Tag: cp311-cp311-manylinux_2_39_x86_64
- Note: use python -m scikit_build_core.builder.wheel_tag for further options
Get Requires: ['cmake>=3.15']
Detected CMake and Ninja (all versions):
* CMake: /usr/bin/cmake <Version('3.28.4')>
* Ninja: /usr/bin/ninja <Version('1.11.1')>
Any ideas on what the problem is?
Issue is with ESPRESSO_INSTALL_PYTHON
and the other variables used in install
in the cmake files. These should point to ./
see https://scikit-build-core.readthedocs.io/en/latest/cmakelists.html#install-directories
I think that’s here: https://github.com/espressomd/espresso/blob/7ad0534debbbe9dac048f4ff322054001de24587/CMakeLists.txt#L266
It’s trying to install into site packages directly instead of the wheel.
Yeah, setting ESPRESSO_INSTALL_PYTHON
to ./
did solve the failing build process, but it cannot find a shared file (espresso_script_interface.so
) when trying to import. However, directly importing the package from within the venvs lib/python3.11/site-packages/
works, any ideas on that?
That is mostly dependent on how the libraries are linked via RPATH
(in the build directory it should be linked appropriately that you can execute them), and after install when the RPATH
is stripped, it depends on what library paths are visible, e.g. via LD_LIBRARY_PATH
. The main issue here is where is espresso_script_interface.so
installed to? You can use CMAKE_INSTALL_RPATH
if you know in advance.
There are 2 more designs you can consider, making espresso_script_interface.a
a static library, or having the python script pre-load the library from a known location. It ultimately depends on the details of the project which approach is most suitable
Thanks a lot for the helpful input! I will discuss this with the maintainers of espresso and leave this issue open until we have come to a conclusion.
I was able to resolve this issue by properly setting the RPATH
of the shared object files using CMakes Development environment, now every .so
is found. Thanks a lot for the help!