Use maturin container (or even action) for manylinux builds
pechersky opened this issue · comments
Thanks for proposing a new feature!
Motivation
The current builds (https://github.com/wasmerio/wasmer-python/actions/runs/4852776764/job/13149022773#step:18:11) are at a high ABI: manylinux_2_34
. This is much higher than ABIs that are still supported, like manylinux_2_17
. Switching to using an existing maturin
container will provide a build context that already supplies Python and Rust.
Proposed solution
Here is the workflow for building the api and the cranelift (as an example):
build:
name: Build and Release using a container
strategy:
matrix:
python: ["3.7", "3.8", "3.9", "3.10", "3.11"]
target:
- id: 'linux-amd64'
os: 'ubuntu-latest'
target-name: 'x86_64-unknown-linux-gnu'
rust-toolchain: 'stable'
fail-fast: true
runs-on: ${{ matrix.target.os }}
container:
image: ghcr.io/pyo3/maturin
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Set up Rust
run: rustup default ${{ matrix.target.rust-toolchain }}
- name: Build api
run: |
pip${{ matrix.python }} install virtualenv
virtualenv .env
source .env/bin/activate
maturin build --bindings pyo3 --release --target ${{ matrix.target.target-name }} --strip --interpreter ${{ matrix.python }} -m packages/api/Cargo.toml
- name: Build cranelist
run: |
pip${{ matrix.python }} install virtualenv
virtualenv .env
source .env/bin/activate
maturin build --bindings pyo3 --release --target ${{ matrix.target.target-name }} --strip --interpreter ${{ matrix.python }} -m packages/compiler-cranelift/Cargo.toml
- name: Upload artifacts
uses: actions/upload-artifact@v2
with:
name: wheels-${{ matrix.python }}-${{ matrix.target.id }}
path: target/wheels
if-no-files-found: error
retention-days: 1
Some additional work in necessary to make LLVM builds installable.
Alternatives
This allows the maturin
team to maintain the containers/toolkit combinations that allow a broad manylinux
build possible. Alternatives are switching to a different context without using the container. This means, in either case, rewriting the justfile
.
Additional context
Some of the cloud containers out there are on ABIs that are too low for manylinux_2_34
. Having broader ABIs allows installing these packages in those context too.