Could you please include a setup.py file?

[mvsoliveira]

Hi,

First, many thanks for developing the bigtree, it is very useful for a project I am working on at CERN.

I created a yocto recipe to install bigtree automatically in ARM-based SoCs. However, the yocto recipe mechanism expects a setup.py file in the root of your repo. I can see you have a pyproject.toml but not a setup.py. Other packages like pandas and numpy have both, so the yocto build work without problems for both packages.

Would you kindly agree on adding a setup.py file to your repository?

Please see enclosed the error I am getting:

NOTE: Executing Tasks
NOTE: Setscene tasks completed
ERROR: python3-bigtree-0.9.4-r0 do_configure: Execution of '/home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/temp/run.do_configure.33981' failed with exit code 2:
/home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/recipe-sysroot-native/usr/bin/python3-native/python3: can't open file 'setup.py': [Errno 2] No such file or directory
WARNING: /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/temp/run.do_configure.33981:1 exit 2 from 'NO_FETCH_BUILD=1 /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/recipe-sysroot-native/usr/bin/python3-native/python3 setup.py clean'

ERROR: Logfile of failure stored in: /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/temp/log.do_configure.33981
Log data follows:
| DEBUG: Executing shell function do_configure
| /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/recipe-sysroot-native/usr/bin/python3-native/python3: can't open file 'setup.py': [Errno 2] No such file or directory
| WARNING: /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/temp/run.do_configure.33981:1 exit 2 from 'NO_FETCH_BUILD=1 /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/recipe-sysroot-native/usr/bin/python3-native/python3 setup.py clean'
| ERROR: Execution of '/home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/temp/run.do_configure.33981' failed with exit code 2:
| /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/recipe-sysroot-native/usr/bin/python3-native/python3: can't open file 'setup.py': [Errno 2] No such file or directory
| WARNING: /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/temp/run.do_configure.33981:1 exit 2 from 'NO_FETCH_BUILD=1 /home/msilvaol/gfexos4/xilinx_bitbake/build/tmp/work/aarch64-xilinx-linux/python3-bigtree/0.9.4-r0/recipe-sysroot-native/usr/bin/python3-native/python3 setup.py clean'
| 
ERROR: Task (/home/msilvaol/gfexos4/xilinx_bitbake/sources/meta-l1calo/recipes-python/python3-bigtree_0.9.4.bb:do_configure) failed with exit code '1'

[Kay Jan W.]

Hi,

Thanks for using bigtree! This seems to be a compatibility issue with legacy builds according to setuptools documentation as pyproject.toml files are becoming the standard way of specifying project metadata.

I have looked into the possibility of adding a setup.py file. The current build-backend that bigtree is using is hatch which has instructions for migrating setup.py file to pyproject.toml file but does not seem to have examples where both files exist at the same time.

For numpy and pandas python packages, the build-backend is using meson-python and according to this comparison article, it seems like poetry and hatch can do much more than meson-python or setuptools.

I also looked into popular Python packages such as fastapi, which is also using hatch as a build-backend and the codebase contains only pyproject.toml file as well.

On another note, I'm not sure if this solution can help your issue with yocto image. Happy to discuss further if there are alternative solutions!

[mvsoliveira]

Hi Kay,

Thanks for the explanation and proposed solution. I found many proposed solutions that didn't work for me but I hadn't found the same page as you did. Choosing the right search keywords is an art :) An adaptation of the solution you proposed worked for me. I am adding here the full recipe as a reference if someone has the same issue in the future. It is running against yocto version zeus.

SUMMARY = "BigTree - Tree Implementation for Python, integrated with Python list, dictionary, and pandas DataFrame."
HOMEPAGE = "https://github.com/kayjan/bigtree"
AUTHOR = "Kay Jan"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=d17cca707393ef99291b58377ab60797"

PYPI_PACKAGE = "bigtree"

inherit pypi setuptools3

SRC_URI = "https://files.pythonhosted.org/packages/a3/06/d26933e315119b44cb2b7cfcfde9085578c4bfefd8991b2f226ab875ec4e/bigtree-0.9.4.tar.gz"

SRC_URI[md5sum] = "ff93e5bfaf3cd40c07a9bf4c9c0c0b34"
SRC_URI[sha256sum] = "37df7ef5b31bb3d81c09ee950b0fd208446fc2f20a2ad9787826deecda68e821"

RDEPENDS_${PN} = "python3-pandas"

do_configure_prepend() {
cat > ${S}/setup.py <<-EOF
from setuptools import setup, find_packages

setup(
       name="${PYPI_PACKAGE}",
       version="${AUTHOR}",
       license="${LICENSE}",
       packages=find_packages(),    
       install_requires=['numpy >= 1.17.0', 'pandas >= 0.23.4'],
)
EOF
}

[Eli Schwartz]

For numpy and pandas python packages, the build-backend is using meson-python and according to this comparison article, it seems like poetry and hatch can do much more than meson-python or setuptools.

Presumably that is because poetry and hatch are build frontends, while meson-python and setuptools are build backends.

poetry-core and hatchling are better comparisons here; they are backends. (And poetry-core in particular is very underpowered since officially you cannot do build steps at all, the only mechanism to do so is -- still -- undocumented.)

I am adding here the full recipe as a reference if someone has the same issue in the future. It is running against yocto version zeus.

And I see the problem. You're trying to inherit setuptools3 when you should be inheriting python_pep517.