`docker build` failed when using `crun` instead of `runc`
akhilerm opened this issue · comments
Description
Using the following Dockerfile
FROM ubuntu:22.04
RUN echo hello
- while running the build command with runc
akhil@akhil-ThinkPad-L14:~/W/d/docker $ docker build --no-cache -t ubuntu:test -f Dockerfile .
[+] Building 4.0s (7/7) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 71B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 3.5s
=> [auth] library/ubuntu:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/2] FROM docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2 0.0s
=> [2/2] RUN echo hello 0.3s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:4be7aaea08fe5dd276d9a8f80bc49a36ef37d26f1b307619f086424ceddad4a1 0.0s
=> => naming to docker.io/library/ubuntu:test
- After switching to
crun
akhil@akhil-ThinkPad-L14:~/W/d/docker $ docker build --no-cache -t ubuntu:test -f Dockerfile .
[+] Building 1.2s (5/5) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 71B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 0.9s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/2] FROM docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2 0.0s
=> ERROR [2/2] RUN echo hello 0.2s
------
> [2/2] RUN echo hello:
0.094 run: unrecognized option '--keep'
0.094 Try `run --help' or `run --usage' for more information.
------
Dockerfile:3
--------------------
1 | FROM ubuntu:22.04
2 |
3 | >>> RUN echo hello
4 |
--------------------
ERROR: failed to solve: process "/bin/sh -c echo hello" did not complete successfully: exit code: 64
But if I am using nerdctl
to do the build, both runc
and crun
works fine. Not sure if its a buildx
or cli
issue, couldnt trace the code part that does this.
Reproduce
- Replace
runc
binary withcrun
. (crun binary should also be renamed runc) - Use the above Dockerfile and build.
Expected behavior
docker build
should succeed.
docker version
1. `docker version` with `runc`
Client: Docker Engine - Community
Version: 26.1.1
API version: 1.45
Go version: go1.21.9
Git commit: 4cf5afa
Built: Tue Apr 30 11:47:53 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 26.1.1
API version: 1.45 (minimum version 1.24)
Go version: go1.21.9
Git commit: ac2de55
Built: Tue Apr 30 11:47:53 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v2.0.0-beta.0-551-ga93eec9bd
GitCommit: a93eec9bda75abc525ffaf0623e4d5e0c155190d
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e946
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker version
withcrun
Client: Docker Engine - Community
Version: 26.1.1
API version: 1.45
Go version: go1.21.9
Git commit: 4cf5afa
Built: Tue Apr 30 11:47:53 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 26.1.1
API version: 1.45 (minimum version 1.24)
Go version: go1.21.9
Git commit: ac2de55
Built: Tue Apr 30 11:47:53 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v2.0.0-beta.0-551-ga93eec9bd
GitCommit: a93eec9bda75abc525ffaf0623e4d5e0c155190d
runc:
Version: 1.14.4
GitCommit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client: Docker Engine - Community
Version: 26.1.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.14.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.27.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 5
Server Version: 26.1.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: a93eec9bda75abc525ffaf0623e4d5e0c155190d
runc version: a220ca661ce078f2c37b38c92e66cf66c012d9c1
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.5.0-28-generic
Operating System: Ubuntu 23.10
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 30.14GiB
Name: akhil-ThinkPad-L14
ID: YJK2:ENQF:MH4T:UDGW:O74Y:M7M7:RDJT:3RXI:UKOQ:PLB6:XQSV:LENZ
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: akhilerm
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional Info
akhil@akhil-ThinkPad-L14:~/W/d/docker $ runc --version
crun version 1.14.4
commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
rundir: /run/user/1000/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
akhil@akhil-ThinkPad-L14:~/W/d/docker $ toggle-runc
Swithced to runc
akhil@akhil-ThinkPad-L14:~/W/d/docker $ runc --version
runc version 1.1.12
commit: v1.1.12-0-g51d5e946
spec: 1.0.2-dev
go: go1.20.13
libseccomp: 2.5.4
But if I am using nerdctl to do the build, both runc and crun works fine. Not sure if it's a buildx or cli issue, couldnt trace the code part that does this.
I think neither; docker/cli
is not handling build on the client side (which is delegated to buildx), and buildx
is only the client; the code related to this is in BuildKit, but it's assuming runc
is the actual OCI runtime in this case, so it likely assumes that the runc
binary provides the --keep
flag. https://github.com/moby/buildkit/blob/51d85d712fad213cd10ac362b18c0a5aab909923/executor/runcexecutor/executor_linux.go#L24-L30
func (w *runcExecutor) run(ctx context.Context, id, bundle string, process executor.ProcessInfo, started func(), keep bool) error {
killer := newRunProcKiller(w.runc, id)
return w.callWithIO(ctx, process, started, killer, func(ctx context.Context, started chan<- int, io runc.IO, pidfile string) error {
extraArgs := []string{}
if keep {
extraArgs = append(extraArgs, "--keep")
}
In your case, crun
is aliased as runc
, so it'll likely still assume it's an actual runc
binary, or a binary with the same ABI. Not sure if it does any feature detection though (and depending on that could set the keep
options or not set it)
I see there was a ticket for this in the BuildKit repository;
And it looks like crun
had a ticket for this, and crun
1.15 added support for the --keep
option; https://github.com/containers/crun/releases/tag/1.15
Thank you for the details. I hadnt checked the buildkit repository for the code. Yep, and the latest crun
fixes this issue. My bad didnt update to that version.
q: Other than aliasing, is there a way to specify to buildx
to use crun
instead of runc
?
q: Other than aliasing, is there a way to specify to buildx to use crun instead of runc?
For the default BuildKit builder built in to the docker daemon, there's not currently an option for that.
Well, there IS an option, but it's currently a temporary one (see moby/moby#47563), so I don't recommend depending on that (as it will go away) we do have an internal tracking ticket to see if we can provide a permanent configuration option for that, but that's not there (yet).
I'm curious though what nerdctl
did to make this work without having to update crun
to a version that supports the --keep
option; it's possible that it's not using the runc
executor, but instead uses the containerd
executor. Perhaps @AkihiroSuda knows from the top of his head.