rootless-containers / usernetes

Kubernetes without the root privileges

Home Page:https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2033-kubelet-in-userns-aka-rootless

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Deriving my own Docker image

dg424 opened this issue · comments

commented

Hi,

I'm trying to build my own image using the main usernetes image as the base image but it doesn't start. The Dockerfile has one line:

FROM ghcr.io/rootless-containers/usernetes:master

Running with the same sample line from the docs:

docker run -td --name usernetes-node -p 127.0.0.1:6443:6443 --privileged usernetes --cri=containerd

Gives the following errors;

Aug 25 14:43:49 1dded8a4ab08 systemd[1]: Started Session c1 of User user.
Aug 25 14:43:49 1dded8a4ab08 systemd[50]: pam_unix(login:session): session opened for user user(uid=1000) by (uid=0)
exit 1
Connection to the local host terminated.
Sending SIGTERM to remaining processes...
Sending SIGKILL to remaining processes...
All filesystems, swaps, loop devices, MD devices and DM devices detached.
Exiting container.

Any ideas ?

Could you provide docker info and docker version

commented

$ docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
compose: Docker Compose (Docker Inc., v2.0.0-beta.6)
scan: Docker Scan (Docker Inc., v0.8.0)

Server:
Containers: 19
Running: 11
Paused: 0
Stopped: 8
Images: 273
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.43.3-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.63GiB
Name: docker-desktop
ID: MBSU:OYE3:SYPP:D2E5:4BWY:XEHE:KV4O:CVUP:UYNQ:34BS:JRYO:JTDR
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 112673
Goroutines: 112665
System Time: 2021-08-26T02:45:06.5133409Z
EventsListeners: 3
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

$ docker version
Client: Docker Engine - Community
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:54:27 2021
OS/Arch: linux/amd64
Context: default
Experimental: true

Server: Docker Engine - Community
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: b0f5bc3
Built: Wed Jun 2 11:54:58 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0-rc95
GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
docker-init:
Version: 0.19.0
GitCommit: de40ad0

Server is docker desktop.

Cgroup Version: 1

Usernetes v20210708.0 is the last version that supports cgroup v1.
The current master requires cgroup v2.
(I don't use WSL and I don't know how to configure WSL to use cgroup v2)

commented

But when I run it directly, not using a derived container, it works:

$ docker run -it --name usernetes-node -p 127.0.0.1:6443:6443 --
privileged ghcr.io/rootless-containers/usernetes --cri=containerd
Created symlink /etc/systemd/system/systemd-firstboot.service → /dev/null.
Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
Created symlink /etc/systemd/system/multi-user.target.wants/docker-entrypoint.service → /etc/systemd/system/docker-entrypoint.service.
/docker-entrypoint.sh: starting /lib/systemd/systemd --show-status=false --unit=docker-entrypoint.target
systemd v248.3-1.fc34 running in system mode. (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Detected virtualization wsl.
Detected architecture x86-64.
Failed to create symlink /sys/fs/cgroup/net_cls: File exists
Failed to create symlink /sys/fs/cgroup/net_prio: File exists
Failed to create symlink /sys/fs/cgroup/cpuacct: File exists
Failed to create symlink /sys/fs/cgroup/cpu: File exists
modprobe@configfs.service: Deactivated successfully.
modprobe@drm.service: Deactivated successfully.
modprobe@fuse.service: Deactivated successfully.
+ source /etc/docker-entrypoint-cmd
++ unsudo /home/user/usernetes/boot/docker-2ndboot.sh --cri=containerd
+ car=/home/user/usernetes/boot/docker-2ndboot.sh
+ shift
+ cdr=--cri=containerd
++ which /home/user/usernetes/boot/docker-2ndboot.sh
+ exec machinectl shell user@ /home/user/usernetes/boot/docker-2ndboot.sh --cri=containerd
Connected to the local host. Press ^] three times within 1s to exit session.
+ ./install.sh --cri=containerd
[WARNING] Disabling Rootless cgroup: the system is using cgroup v1, you need to reboot the system with systemd.unified_cgroup_hierarchy=1
[WARNING] Cgroup is disabled. In future version of Usernetes, cgroup (v2) will be an essential requirement.
[INFO] Generating single-node cluster TLS keys (/home/user/.config/usernetes/{master,node})
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{ca.pem,ca-key.pem}
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{admin.pem,admin-key.pem}
2021/08/27 12:52:52 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/admin-localhost.kubeconfig
Cluster "kubernetes-the-hard-way" set.
User "admin" set.
Context "default" created.
Switched to context "default".
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/admin-127.0.0.1.kubeconfig
Cluster "kubernetes-the-hard-way" set.
User "admin" set.
Context "default" created.
Switched to context "default".
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{kube-controller-manager.pem,kube-controller-manager-key.pem}
2021/08/27 12:52:53 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/kube-controller-manager.kubeconfig
Cluster "kubernetes-the-hard-way" set.
User "system:kube-controller-manager" set.
Context "default" created.
Switched to context "default".
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{kube-proxy.pem,kube-proxy-key.pem}
2021/08/27 12:52:54 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/kube-proxy.kubeconfig
Cluster "kubernetes-the-hard-way" set.
User "system:kube-proxy" set.
Context "default" created.
Switched to context "default".
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{kube-scheduler.pem,kube-scheduler-key.pem}
2021/08/27 12:52:54 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/kube-scheduler.kubeconfig
Cluster "kubernetes-the-hard-way" set.
User "system:kube-scheduler" set.
Context "default" created.
Switched to context "default".
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{kubernetes.pem,kubernetes-key.pem}
[INFO] Creating /tmp/cfssl.iEzfftKA3/master/{service-account.pem,service-account-key.pem}
2021/08/27 12:52:55 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[INFO] Writing 127.0.0.1 to /tmp/cfssl.iEzfftKA3/nodes.2515bdb6e5e8/master
[INFO] Copying /tmp/cfssl.iEzfftKA3/master/ca.pem to /tmp/cfssl.iEzfftKA3/nodes.2515bdb6e5e8/ca.pem
[INFO] Creating /tmp/cfssl.iEzfftKA3/nodes.2515bdb6e5e8/{node.pem,node-key.pem}
[INFO] Copying /tmp/cfssl.iEzfftKA3/master/kube-proxy.kubeconfig to /tmp/cfssl.iEzfftKA3/nodes.2515bdb6e5e8/kube-proxy.kubeconfig
[INFO] Creating /tmp/cfssl.iEzfftKA3/nodes.2515bdb6e5e8/node.kubeconfig
Cluster "kubernetes-the-hard-way" set.
User "system:node:2515bdb6e5e8" set.
Context "default" created.
Switched to context "default".
[INFO] Base dir: /home/user/usernetes
[INFO] Installing /home/user/.config/systemd/user/u7s.target
[INFO] Installing /home/user/.config/systemd/user/u7s-master-with-etcd.target
[INFO] Installing /home/user/.config/systemd/user/u7s-rootlesskit.service
[INFO] Installing /home/user/.config/systemd/user/u7s-etcd.target
[INFO] Installing /home/user/.config/systemd/user/u7s-etcd.service
[INFO] Installing /home/user/.config/systemd/user/u7s-master.target
[INFO] Installing /home/user/.config/systemd/user/u7s-kube-apiserver.service
[INFO] Installing /home/user/.config/systemd/user/u7s-kube-controller-manager.service
[INFO] Installing /home/user/.config/systemd/user/u7s-kube-scheduler.service
[INFO] Installing /home/user/.config/systemd/user/u7s-node.target
[INFO] Installing /home/user/.config/systemd/user/u7s-containerd-fuse-overlayfs-grpc.service
[INFO] Installing /home/user/.config/systemd/user/u7s-kubelet-containerd.service
[INFO] Installing /home/user/.config/systemd/user/u7s-kube-proxy.service
[INFO] Starting u7s.target
+ systemctl --user -T enable u7s.target
Created symlink /home/user/.config/systemd/user/multi-user.target.wants/u7s.target → /home/user/.config/systemd/user/u7s.target.
+ systemctl --user -T start u7s.target
Enqueued anchor job 12 u7s.target/start.
Enqueued auxiliary job 32 u7s-kube-scheduler.service/start.
Enqueued auxiliary job 29 u7s-master.target/start.
Enqueued auxiliary job 30 u7s-kube-apiserver.service/start.
Enqueued auxiliary job 27 u7s-kube-proxy.service/start.
Enqueued auxiliary job 25 u7s-rootlesskit.service/start.
Enqueued auxiliary job 33 u7s-kube-controller-manager.service/start.
Enqueued auxiliary job 28 u7s-master-with-etcd.target/start.
Enqueued auxiliary job 14 u7s-containerd-fuse-overlayfs-grpc.service/start.
Enqueued auxiliary job 34 u7s-etcd.target/start.
Enqueued auxiliary job 13 u7s-node.target/start.
Enqueued auxiliary job 26 u7s-kubelet-containerd.service/start.
Enqueued auxiliary job 31 u7s-etcd.service/start.

real    0m5.706s
user    0m0.000s
sys     0m0.011s
+ systemctl --user --all --no-pager list-units 'u7s-*'
  UNIT                                       LOAD   ACTIVE SUB     DESCRIPTION
  u7s-containerd-fuse-overlayfs-grpc.service loaded active running Usernetes containerd-fuse-overlayfs-grpc service
  u7s-etcd.service                           loaded active running Usernetes etcd service
  u7s-kube-apiserver.service                 loaded active running Usernetes kube-apiserver service
  u7s-kube-controller-manager.service        loaded active running Usernetes kube-controller-manager service
  u7s-kube-proxy.service                     loaded active running Usernetes kube-proxy service
  u7s-kube-scheduler.service                 loaded active running Usernetes kube-scheduler service
  u7s-kubelet-containerd.service             loaded active running Usernetes kubelet service (containerd)
  u7s-rootlesskit.service                    loaded active running Usernetes RootlessKit service (containerd)
  u7s-etcd.target                            loaded active active  Usernetes target for etcd
  u7s-master-with-etcd.target                loaded active active  Usernetes target for Kubernetes master components (i…
  u7s-master.target                          loaded active active  Usernetes target for Kubernetes master components
  u7s-node.target                            loaded active active  Usernetes target for Kubernetes node components (con…

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
12 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.
+ set +x
[INFO] Hint: `sudo loginctl enable-linger` to start user services automatically on the system start up.
[INFO] Hint: To enable addons including CoreDNS, run: kubectl apply -f /home/user/usernetes/manifests/*.yaml
[INFO] Hint: export KUBECONFIG=/home/user/.config/usernetes/master/admin-localhost.kubeconfig
$ uname -a
Linux N-20HJPF19TVVM 5.10.43.3-microsoft-standard-WSL2 #1 SMP Thu Jul 8 14:40:50 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
$ grep cgroup /proc/filesystems
nodev   cgroup
nodev   cgroup2

Because the latest tag is set to v20210708.0, which still supports cgroup v1.