Docker build fails due to ignored directories
MattTimms opened this issue · comments
There's conflict between the Dockerfile & its .dockerignore that causes docker build to fail, i.e. make docker-build with: (full terminal output at the end)
30.10 error: failed to load manifest for workspace member `/lychee/examples/*`
30.10
30.10 Caused by:
30.10 failed to read `/lychee/examples/*/Cargo.toml`
30.10
30.10 Caused by:
30.11 No such file or directory (os error 2)The contents of .dockerignore include:
examples
benches
Which are both required in the docker build process.
I understand omitting these directories from lychee's docker image, but I'm not a competent enough rust developer to suggest the changes needed to remedy this issue besides removing entries from the .dockerignore file.
Full command:
> make docker-build
docker build -t "lycheeverse/lychee" .
[+] Building 31.5s (15/16) docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.44kB 0.0s
=> [internal] load metadata for docker.io/library/debian:bullseye-slim 1.1s
=> [internal] load metadata for docker.io/library/rust:latest 1.1s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 178B 0.0s
=> [builder 1/8] FROM docker.io/library/rust:latest@sha256:00e330d2e2cdada2b75e9517c8359df208b3c880c5e34cb802c120083d5 0.0s
=> [stage-1 1/3] FROM docker.io/library/debian:bullseye-slim@sha256:a165446a88794db4fec31e35e9441433f9552ae048fb1ed26d 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 5.57kB 0.0s
=> CACHED [stage-1 2/3] RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-r 0.0s
=> CACHED [builder 2/8] RUN USER=root cargo new --bin lychee 0.0s
=> CACHED [builder 3/8] WORKDIR /lychee 0.0s
=> CACHED [builder 4/8] COPY lychee-bin/Cargo.toml lychee-bin/Cargo.toml 0.0s
=> CACHED [builder 5/8] COPY lychee-lib/Cargo.toml lychee-lib/Cargo.toml 0.0s
=> CACHED [builder 6/8] RUN cargo build --release && rm src/*.rs 0.0s
=> CACHED [builder 7/8] COPY . ./ 0.0s
=> ERROR [builder 8/8] RUN rm ./target/release/deps/lychee* && cargo build --release 30.3s
------
> [builder 8/8] RUN rm ./target/release/deps/lychee* && cargo build --release:
0.612 info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
1.129 info: latest update on 2024-03-21, rust version 1.77.0 (aedd173a2 2024-03-17)
1.129 info: downloading component 'cargo'
2.399 info: downloading component 'clippy'
2.750 info: downloading component 'rust-docs'
5.235 info: downloading component 'rust-std'
9.705 info: downloading component 'rustc'
20.40 info: downloading component 'rustfmt'
20.77 info: installing component 'cargo'
21.33 info: installing component 'clippy'
21.57 info: installing component 'rust-docs'
24.52 info: installing component 'rust-std'
26.16 info: installing component 'rustc'
29.76 info: installing component 'rustfmt'
30.10 error: failed to load manifest for workspace member `/lychee/examples/*`
30.10
30.10 Caused by:
30.10 failed to read `/lychee/examples/*/Cargo.toml`
30.10
30.10 Caused by:
30.11 No such file or directory (os error 2)
------
Dockerfile:20
--------------------
19 | COPY . ./
20 | >>> RUN rm ./target/release/deps/lychee* \
21 | >>> && cargo build --release
22 |
--------------------
ERROR: failed to solve: process "/bin/sh -c rm ./target/release/deps/lychee* && cargo build --release" did not complete successfully: exit code: 101
make: *** [Makefile:11: docker-build] Error 1
Huh!
We build the image with every release (with every pull request, even), so I wonder how this did break?
In general, it would be wise to exclude examples and benchmarks from the image and fix the release build.
Perhaps it's as easy as removing "examples/*", "benches" from
Line 2 in d5ddf66
Can anybody test if examples (e.g. cargo run --example builder) and cargo bench would still work in this case? 😃
Perhaps it's as easy as removing "examples/*", "benches"
Can anybody test if examples (e.g. cargo run --example builder) and cargo bench would still work in this case? 😃
I'm afraid not:
> cargo run --example builder
error: no example target named `builder`.
> cargo bench
...
test result: ok. 0 passed; 0 failed; 198 ignored; 0 measured; 0 filtered out; finished in 0.00sWe build the image with every release (with every pull request, even), so I wonder how this did break?
I think that is because the docker image pushed to docker.io, as part of the GitHub Actions CI, are the images created with Dockerfile-CI*.Dockerfile which fetch executables from GitHub releases, rather than the Dockerfile that I've been focused on which builds from src.
In general, it would be wise to exclude examples and benchmarks from the image and fix the release build.
I agree, typically you'd exclude such. I tried alternatives such as selective package building (i.e. cargo build --package ...) & default-members/members edits to the root Cargo.toml, but looks like since these packages are defined in the cargo file it will complain when it cannot see them on the filesystem/within docker.
That all being said, since the lychee executable is the only thing copied across from the builder stage, perhaps there's little harm in allowing examples/ & benches/?
I agree, it seems like the simplest solution and it does not affect the final docker image or its size. Also I don't think that this should slow down the build in any noticeable way.
Thanks for looking into it @MattTimms, and thanks for the feedback @thomas-zahner. 🙏