docker / buildx

Docker CLI plugin for extended build capabilities with BuildKit

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`docker build` failed when using `crun` instead of `runc`

akhilerm opened this issue · comments

Description

Using the following Dockerfile

FROM ubuntu:22.04
RUN echo hello
  • while running the build command with runc
akhil@akhil-ThinkPad-L14:~/W/d/docker $ docker build  --no-cache -t ubuntu:test -f Dockerfile .
[+] Building 4.0s (7/7) FINISHED                                                                                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                     0.0s
 => => transferring dockerfile: 71B                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                          3.5s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                          0.0s
 => CACHED [1/2] FROM docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2                                                             0.0s
 => [2/2] RUN echo hello                                                                                                                                                                 0.3s
 => exporting to image                                                                                                                                                                   0.1s
 => => exporting layers                                                                                                                                                                  0.0s
 => => writing image sha256:4be7aaea08fe5dd276d9a8f80bc49a36ef37d26f1b307619f086424ceddad4a1                                                                                             0.0s
 => => naming to docker.io/library/ubuntu:test       
  • After switching to crun
akhil@akhil-ThinkPad-L14:~/W/d/docker $ docker build  --no-cache -t ubuntu:test -f Dockerfile .
[+] Building 1.2s (5/5) FINISHED                                                                                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                     0.0s
 => => transferring dockerfile: 71B                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                          0.9s
 => [internal] load .dockerignore                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                          0.0s
 => CACHED [1/2] FROM docker.io/library/ubuntu:22.04@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2                                                             0.0s
 => ERROR [2/2] RUN echo hello                                                                                                                                                           0.2s
------                                                                                                                                                                                        
 > [2/2] RUN echo hello:                                                                                                                                                                      
0.094 run: unrecognized option '--keep'
0.094 Try `run --help' or `run --usage' for more information.
------
Dockerfile:3
--------------------
   1 |     FROM ubuntu:22.04
   2 |     
   3 | >>> RUN echo hello
   4 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c echo hello" did not complete successfully: exit code: 64

But if I am using nerdctl to do the build, both runc and crun works fine. Not sure if its a buildx or cli issue, couldnt trace the code part that does this.

Reproduce

  1. Replace runc binary with crun. (crun binary should also be renamed runc)
  2. Use the above Dockerfile and build.

Expected behavior

docker build should succeed.

docker version

1. `docker version` with `runc`

Client: Docker Engine - Community
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:47:53 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:47:53 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v2.0.0-beta.0-551-ga93eec9bd
  GitCommit:        a93eec9bda75abc525ffaf0623e4d5e0c155190d
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e946
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
  1. docker version with crun
Client: Docker Engine - Community
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:47:53 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:47:53 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v2.0.0-beta.0-551-ga93eec9bd
  GitCommit:        a93eec9bda75abc525ffaf0623e4d5e0c155190d
 runc:
  Version:          1.14.4
  GitCommit:        a220ca661ce078f2c37b38c92e66cf66c012d9c1
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    26.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 5
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: a93eec9bda75abc525ffaf0623e4d5e0c155190d
 runc version: a220ca661ce078f2c37b38c92e66cf66c012d9c1
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-28-generic
 Operating System: Ubuntu 23.10
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.14GiB
 Name: akhil-ThinkPad-L14
 ID: YJK2:ENQF:MH4T:UDGW:O74Y:M7M7:RDJT:3RXI:UKOQ:PLB6:XQSV:LENZ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: akhilerm
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

akhil@akhil-ThinkPad-L14:~/W/d/docker $ runc --version
crun version 1.14.4
commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
rundir: /run/user/1000/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
akhil@akhil-ThinkPad-L14:~/W/d/docker $ toggle-runc 
Swithced to runc
akhil@akhil-ThinkPad-L14:~/W/d/docker $ runc --version
runc version 1.1.12
commit: v1.1.12-0-g51d5e946
spec: 1.0.2-dev
go: go1.20.13
libseccomp: 2.5.4

But if I am using nerdctl to do the build, both runc and crun works fine. Not sure if it's a buildx or cli issue, couldnt trace the code part that does this.

I think neither; docker/cli is not handling build on the client side (which is delegated to buildx), and buildx is only the client; the code related to this is in BuildKit, but it's assuming runc is the actual OCI runtime in this case, so it likely assumes that the runc binary provides the --keep flag. https://github.com/moby/buildkit/blob/51d85d712fad213cd10ac362b18c0a5aab909923/executor/runcexecutor/executor_linux.go#L24-L30

func (w *runcExecutor) run(ctx context.Context, id, bundle string, process executor.ProcessInfo, started func(), keep bool) error {
	killer := newRunProcKiller(w.runc, id)
	return w.callWithIO(ctx, process, started, killer, func(ctx context.Context, started chan<- int, io runc.IO, pidfile string) error {
		extraArgs := []string{}
		if keep {
			extraArgs = append(extraArgs, "--keep")
		}

In your case, crun is aliased as runc, so it'll likely still assume it's an actual runc binary, or a binary with the same ABI. Not sure if it does any feature detection though (and depending on that could set the keep options or not set it)

I see there was a ticket for this in the BuildKit repository;

And it looks like crun had a ticket for this, and crun 1.15 added support for the --keep option; https://github.com/containers/crun/releases/tag/1.15

Thank you for the details. I hadnt checked the buildkit repository for the code. Yep, and the latest crun fixes this issue. My bad didnt update to that version.

q: Other than aliasing, is there a way to specify to buildx to use crun instead of runc?

q: Other than aliasing, is there a way to specify to buildx to use crun instead of runc?

For the default BuildKit builder built in to the docker daemon, there's not currently an option for that.

Well, there IS an option, but it's currently a temporary one (see moby/moby#47563), so I don't recommend depending on that (as it will go away) we do have an internal tracking ticket to see if we can provide a permanent configuration option for that, but that's not there (yet).

I'm curious though what nerdctl did to make this work without having to update crun to a version that supports the --keep option; it's possible that it's not using the runc executor, but instead uses the containerd executor. Perhaps @AkihiroSuda knows from the top of his head.