Giters

wyfo / apischema

JSON (de)serialization, GraphQL and JSON schema generation using Python typing.

Home Page:https://wyfo.github.io/apischema/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

CD workflow to build wheels

wyfo opened this issue · comments

commented

apischema new binary extension requires to build wheels on different platform/architecture/Python version/etc.

It will require a new CD workflow, which could also be used to upload wheels on Pypi, create the Github release, etc.

commented

I've found cibuildwheel very flexible, it gives you hooks to do things like test the installed built wheel on multiple platforms. Here's an example of how we use it: https://github.com/dls-controls/pythonSoftIOC/blob/master/.github/workflows/code.yml

commented

Also, will the C extension be optional? We are planning to use apischema on an embedded platform where the current performance is fine, but cross-compilation of a binary wheel would be difficult...

commented

I've found cibuildwheel very flexible, it gives you hooks to do things like test the installed built wheel on multiple platforms. Here's an example of how we use it: https://github.com/dls-controls/pythonSoftIOC/blob/master/.github/workflows/code.yml

Thanks, I will take a look at it.

Also, will the C extension be optional? We are planning to use apischema on an embedded platform where the current performance is fine, but cross-compilation of a binary wheel would be difficult...

I don't really understand (but I'm quite noob in Python packaging). Why would you have to do cross-compilation?
If apischema does not have a wheel for your platform, it should be installed from source, so C files will be compiled automatically (it takes only a few seconds, but I admit I don't know what happens if there is no C compiler available). Therefore, it should not impact the packaging of your code, as far as I understand Python packaging, but I may be wrong.

commented

Why would you have to do cross-compilation?

On an embedded system (Xilinx Zynq) we cross-compile from a linux-x86_64 host system to an arm target image. We do this first for the linux kernel, then for the busybox-based rootfs, then for all the applications. We can cross-compile wheels (with a bit more effort) when building the image, but once it is running there is no C compiler installed on the system, so you cannot install new libraries with C extensions.

For instance, we wanted to try out tornado without rebuilding the base image, and this works because it has an optional C extension:
https://github.com/tornadoweb/tornado/blob/master/setup.py#L40-L111

I see you already turn off the cythonization for PyPy, it would be nice to allow it to fail like the extension does in tornado if there is no compiler installed...

commented

For instance, we wanted to try out tornado without rebuilding the base image, and this works because it has an optional C extension:
https://github.com/tornadoweb/tornado/blob/master/setup.py#L40-L111

Actually, this is exactly what I wanted to have! but I didn't really know how to do (and I hate Python packaging, so I did not look deeper into it).
I will create an issue and make the compilation optional before releasing it.