extra-container on non-NixOS

Question

extra-container on non-NixOS

danbst opened this issue 6 years ago · comments

I'm documenting here what had to be done to launch container via extra-container on Ubuntu 17.10

First, install from this repo doesn't work. I expected nix-env -if . to work here
Running extra-container requires sudo. But due to bad sudo+Nix integration in Ubuntu, the command to run is more complicated:

$ sudo -E env PATH="$PATH" extra-container create path/to/container --start

I've made an alias alias sudo-extra-container='sudo -E env PATH="$PATH" extra-container' which solved a problem for me, but that wasn't obvious.
3. It depends on nixos-container, but I don't see reasons why it should. For example, restart command is handled by extra-container, but start is delegated to nixos-container. It is possible to add nixos-container as propagated build input, but nixos-container doesn't work great on non-NixOS (actually, doesn't work but I know folks did use it somehow).
4. The /etc/systemd-mutable should be /usr/lib/systemd/system on non-NixOS (see https://www.freedesktop.org/software/systemd/man/systemd.unit.html). Without that containers are not registered in systemd.

Danylo Hlynskyi commented 5 years ago

PR: #7

Danylo Hlynskyi · Answer 1 · Tue Oct 02 2018 17:37:56 GMT+0800 (China Standard Time)

Need create /etc/static/os-release file.

# mkdir /etc/static
# touch /etc/static/os-release

This is buggy situation in nixos container launch script (but it is invisible on NixOS, because this file already exists):

during start, script checks if "$root/etc/os-release" exists
https://github.com/NixOS/nixpkgs/blob/cad1c18743699fa7458f1e49f6cfab0b86b024e9/nixos/modules/virtualisation/containers.nix#L73
This resolves to path /var/lib/containers/CONTAINER_NAME/etc/os-release
if this is first start, check succeeds and file is created
however nixos activation rewrites this file with a symlink to /etc/static/os-release, due to https://github.com/NixOS/nixpkgs/blob/e144899b7492d8fdc48c685516347ba7788245a5/nixos/modules/misc/version.nix#L93
when container restarts, the check still succeeds, but only because link is broken:

# ls -la /var/lib/containers/CONTAINER_NAME/etc/os-release
lrwxrwxrwx 1 root root 22 Oct  2 05:02 /var/lib/containers/CONTAINER_NAME/etc/os-release -> /etc/static/os-release

The link will work after bind-mount, but until bind mount active, it points to /etc/static/os-release in host system. NixOS contains that file, but other distros don't. [ -e ... ] detects invalid symlink as no file and succeeds

touch ... then fails

Oct 02 05:03:21 host systemd[1]: Stopped Container 'CONTAINER_NAME'.
Oct 02 05:03:21 host systemd[1]: Starting Container 'CONTAINER_NAME'...
Oct 02 05:03:21 host container CONTAINER_NAME[31290]: touch: cannot touch '/var/lib/containers/CONTAINER_NAME/etc/os-release': No such file or directory
Oct 02 05:03:21 host systemd[1]: container@CONTAINER_NAME.service: Main process exited, code=exited, status=1/FAILURE
Oct 02 05:03:21 host systemd[1]: Failed to start Container 'CONTAINER_NAME'.
Oct 02 05:03:21 host systemd[1]: container@CONTAINER_NAME.service: Unit entered failed state.
Oct 02 05:03:21 host systemd[1]: container@CONTAINER_NAME.service: Failed with result 'exit-code'.

Danylo Hlynskyi · Answer 2 · Wed Jan 02 2019 22:25:26 GMT+0800 (China Standard Time)

The /etc/os-release issue should be nicely solved by NixOS/nixpkgs#35364

tomberek · Answer 3 · Wed Mar 27 2019 07:28:08 GMT+0800 (China Standard Time)

May also require a touch /nix/var/nix/daemon-socket if that doesn't exist for you.

erikarvstedt · Answer 4 · Sun Nov 01 2020 22:40:57 GMT+0800 (China Standard Time)

Implemented in release 0.5.