2.4: Themes are not installed, leading to failing tests
dvzrv opened this issue · comments
Hi! I'm packaging this project for Arch Linux. This is the build script we are using for 2.3 can be found here: https://gitlab.archlinux.org/archlinux/packaging/packages/sigal/-/blob/1f8a26e7b63f488b70085c6e23e73933cb9162a3/PKGBUILD
The current one does not apply the now irrelevant patch any longer and builds from a github provided source tarball.
Additionally, for the tests I am now installing sigal to a temporary location to test against (more below).
When trying to build 2.4 I noticed that themes directories are no longer installed, which has the tests fail:
E Exception: Impossible to find the theme /build/sigal/src/sigal-2.4/test_dir/usr/lib/python3.11/site-packages/sigal/themes/colorbox
Full build and test logs:
sigal-2.4-1-x86_64-build.log
sigal-2.4-1-x86_64-check.log
The directory structure created by installing sigal is:
/var/lib/archbuild/extra-testing-x86_64/dave-5/build/sigal/src/sigal-2.4/test_dir
└── usr
├── bin
│ └── sigal
└── lib
└── python3.11
└── site-packages
├── sigal
│ ├── __init__.py
│ ├── __main__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-311.opt-1.pyc
│ │ ├── __init__.cpython-311.pyc
│ │ ├── __main__.cpython-311.opt-1.pyc
│ │ ├── __main__.cpython-311.pyc
│ │ ├── gallery.cpython-311.opt-1.pyc
│ │ ├── gallery.cpython-311.pyc
│ │ ├── image.cpython-311.opt-1.pyc
│ │ ├── image.cpython-311.pyc
│ │ ├── log.cpython-311.opt-1.pyc
│ │ ├── log.cpython-311.pyc
│ │ ├── settings.cpython-311.opt-1.pyc
│ │ ├── settings.cpython-311.pyc
│ │ ├── signals.cpython-311.opt-1.pyc
│ │ ├── signals.cpython-311.pyc
│ │ ├── utils.cpython-311.opt-1.pyc
│ │ ├── utils.cpython-311.pyc
│ │ ├── version.cpython-311.opt-1.pyc
│ │ ├── version.cpython-311.pyc
│ │ ├── video.cpython-311.opt-1.pyc
│ │ ├── video.cpython-311.pyc
│ │ ├── writer.cpython-311.opt-1.pyc
│ │ └── writer.cpython-311.pyc
│ ├── gallery.py
│ ├── image.py
│ ├── log.py
│ ├── plugins
│ │ ├── __init__.py
│ │ ├── __pycache__
│ │ │ ├── __init__.cpython-311.opt-1.pyc
│ │ │ ├── __init__.cpython-311.pyc
│ │ │ ├── adjust.cpython-311.opt-1.pyc
│ │ │ ├── adjust.cpython-311.pyc
│ │ │ ├── compress_assets.cpython-311.opt-1.pyc
│ │ │ ├── compress_assets.cpython-311.pyc
│ │ │ ├── copyright.cpython-311.opt-1.pyc
│ │ │ ├── copyright.cpython-311.pyc
│ │ │ ├── extended_caching.cpython-311.opt-1.pyc
│ │ │ ├── extended_caching.cpython-311.pyc
│ │ │ ├── feeds.cpython-311.opt-1.pyc
│ │ │ ├── feeds.cpython-311.pyc
│ │ │ ├── media_page.cpython-311.opt-1.pyc
│ │ │ ├── media_page.cpython-311.pyc
│ │ │ ├── nomedia.cpython-311.opt-1.pyc
│ │ │ ├── nomedia.cpython-311.pyc
│ │ │ ├── nonmedia_files.cpython-311.opt-1.pyc
│ │ │ ├── nonmedia_files.cpython-311.pyc
│ │ │ ├── titleregexp.cpython-311.opt-1.pyc
│ │ │ ├── titleregexp.cpython-311.pyc
│ │ │ ├── upload_s3.cpython-311.opt-1.pyc
│ │ │ ├── upload_s3.cpython-311.pyc
│ │ │ ├── watermark.cpython-311.opt-1.pyc
│ │ │ ├── watermark.cpython-311.pyc
│ │ │ ├── zip_gallery.cpython-311.opt-1.pyc
│ │ │ └── zip_gallery.cpython-311.pyc
│ │ ├── adjust.py
│ │ ├── compress_assets.py
│ │ ├── copyright.py
│ │ ├── encrypt
│ │ │ ├── __init__.py
│ │ │ ├── __pycache__
│ │ │ │ ├── __init__.cpython-311.opt-1.pyc
│ │ │ │ ├── __init__.cpython-311.pyc
│ │ │ │ ├── encrypt.cpython-311.opt-1.pyc
│ │ │ │ ├── encrypt.cpython-311.pyc
│ │ │ │ ├── endec.cpython-311.opt-1.pyc
│ │ │ │ └── endec.cpython-311.pyc
│ │ │ ├── encrypt.py
│ │ │ └── endec.py
│ │ ├── extended_caching.py
│ │ ├── feeds.py
│ │ ├── media_page.py
│ │ ├── nomedia.py
│ │ ├── nonmedia_files.py
│ │ ├── titleregexp.py
│ │ ├── upload_s3.py
│ │ ├── watermark.py
│ │ └── zip_gallery.py
│ ├── settings.py
│ ├── signals.py
│ ├── templates
│ │ ├── __pycache__
│ │ │ ├── sigal.conf.cpython-311.opt-1.pyc
│ │ │ └── sigal.conf.cpython-311.pyc
│ │ └── sigal.conf.py
│ ├── utils.py
│ ├── version.py
│ ├── video.py
│ └── writer.py
└── sigal-2.4.dist-info
├── LICENSE
├── METADATA
├── RECORD
├── WHEEL
├── entry_points.txt
└── top_level.txt
15 directories, 91 files
Hi,
I don't know what's happening. The theme files are present in the wheel and sdist on pypi.org, and it works for me locally when building the wheel from the .tar.gz. It also works when using the PKGBUILD
==> Making package: sigal 2.4-1 (Sat 30 Sep 2023 10:42:42 PM CEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
-> Found sigal-2.4.tar.gz
==> Validating source files with sha512sums...
sigal-2.4.tar.gz ... Passed
==> Extracting sources...
-> Extracting sigal-2.4.tar.gz with bsdtar
==> Starting build()...
* Getting build dependencies for wheel...
/usr/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:72: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
config = read_configuration(filepath, True, ignore_option_errors, dist)
validate_pyproject.api.load_builtin_plugin defines `tool.distutils` schema
validate_pyproject.api.load_builtin_plugin defines `tool.setuptools` schema
running egg_info
writing src/sigal.egg-info/PKG-INFO
writing dependency_links to src/sigal.egg-info/dependency_links.txt
writing entry points to src/sigal.egg-info/entry_points.txt
writing requirements to src/sigal.egg-info/requires.txt
writing top-level names to src/sigal.egg-info/top_level.txt
listing git files failed - pretending there aren't any
reading manifest file 'src/sigal.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'src/sigal.egg-info/SOURCES.txt'
...
One thing I changed since 2.3 is I removed the MANIFEST, since it's not needed when using setuptools-scm. However when building from the .tar.gz there is not git information, and I get a message in the build stage (above): listing git files failed - pretending there aren't any, message that I cannot see in your log.
I'm building with SETUPTOOLS_SCM_PRETEND_VERSION=$pkgver python -m build --wheel --no-isolation.
We are no longer relying on pypi sdist tarballs because they are not clearly defined and broken more often than not.
I think the issue is with the setuptools configuration (specifically https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#finding-simple-packages), which only finds simple packages. All the themes are not packages or namespaced packages though (they lack __init__.py) and looking at https://setuptools.pypa.io/en/latest/userguide/datafiles.html#include-package-data I believe that the MANIFEST.in would still be required for that (otherwise the auto-generated source tarball can not be used).
It seems the more straight-forward way would be to use https://setuptools.pypa.io/en/latest/userguide/datafiles.html#package-data though (as it doesn't require MANIFEST.in).
I believe #503 to fix this issue
Hm, that appears to not yet fix it entirely.
There also seems to be at least one .js file with sigal/plugins/encrypt/static/decrypt.js that needs packaging as well. Will update the PR.
Okay, with #503 applied I can now build from the auto-generated tarball and all tests pass!