Create a larger build image and pipeline with extra packages
myoung34 opened this issue · comments
It would make sense to have a third image type such:
Base Image
├── Base Image With Extras
│ └── Runner Final (Large)
└── Runner Final (Small)
Where Both Finals are identical but have FROM Base:tag or From Base:tag-large
This would let us have:
Base Image
├── Base Image With Extras
│ ├── Language specific packages (python, node, ruby, etc)
│ ├── Browser testing suites, etc
│ └── Compilation specific packages (gcc, make, etc)
└── Runner Final (Small)
└── No change from current
Would resolve:
References:
Objective
I investigated this. It would not be hard to do this, the question is what should be included in each image.
User stories:
- I'm a user who just wants to run a container for self-hosted runners. I want it to behave as much like github provided runners as possible
- I'm a user who just wants to run a container for self-hosted runners. I don't use anything too exotic and want the expected utilities and frameworks to be available. I plan on using other actions to install most complex dependencies.
- I'm a user who wants the bare minimum image for self-hosted runners. I will use it as a base in my own image that will include exactly the additional dependencies I need.
Current State
Today the project is somewhere between 2 and 3.
Diff
docker-github-actions-runner/Dockerfile.base
Lines 20 to 49 in ac2e2b9
(red is from this repo, green is from the official runner image)
-apt-transport-https
+acl
+aria2
+autoconf
+automake
+binutils
+bison
+brotli
build-essential
-ca-certificates
+bzip2
+coreutils
curl
-dirmngr
-dumb-init
-gettext
-gnupg
-gosu
-gpg-agent
-inetutils-ping
+dbus
+dnsutils
+dpkg
+fakeroot
+file
+flex
+fonts-noto-color-emoji
+ftp
+gnupg2
+haveged
+imagemagick
+iproute2
+iputils-ping
jq
-libcurl4-openssl-dev
+lib32z1
+libc++-dev
+libc++abi-dev
+libcurl4
+libgbm-dev
+libgconf-2-4
+libgsl-dev
+libgtk-3-0
+libmagic-dev
+libmagickcore-dev
+libmagickwand-dev
+libsecret-1-dev
+libsqlite3-dev
+libtool
+libunwind8
+libxkbfile-dev
+libxss1
+libyaml-dev
locales
-lsb-release
-nodejs
+m4
+mediainfo
+mercurial
+net-tools
+netcat
openssh-client
-python3
-python3-pip
-python3-setuptools
+p7zip-full
+p7zip-rar
+parallel
+pass
+patchelf
+pkg-config
+pollinate
+python-is-python3
+rpm
rsync
-software-properties-common
+shellcheck
+sphinxsearch
+sqlite3
+ssh
+sshpass
+subversion
sudo
+swig
tar
+telnet
+texinfo
+time
+tk
+tzdata
unzip
+upx
wget
+xorriso
+xvfb
+xz-utils
zip
-zlib1g-dev
-zstd
+zsyncThis repo also includes:
liblttng-ust0orliblttng-ust1awscligit(from git-core)git-lfsdockerdocker-composev1podmanbuildahskopeogh(github-cli)
All of which are also included in the official github runner image. There aren't many extraneous packages that aren't also needed for us setting up and running the actions runner (e.g. gosu).
Prior Art
Where should we draw the lines between base, default, and extra?
Some prior art to look at are the images suggested by https://github.com/nektos/act/blob/master/IMAGES.md which recommend catthehacker/ubuntu:act-20.04 and catthehacker/ubuntu:full-20.04 as the medium and large images respectively.
catthehacker/ubuntu:full-20.04
According to https://github.com/catthehacker/docker_images#images-available the catthehacker/ubuntu:full-20.04 is a 20GB compressed image that extracts to ~60GB. It is also out of date with the upstream github runner images and doesn't seem to be worth updating automatically due to the amount of upstream changes. I'm really not sure what the health is of the full image.
If we really wanted to provide an image to satisfy user story (1) above, building on top of that image is probably the best idea and comes with the smallest amount of maintenance for this project, but also is likely to be out of date and generate "I tried this and it didn't work" or "this image is too big" issues.
catthehacker/ubuntu:act-20.04
The setup script for this image is https://github.com/catthehacker/docker_images/blob/master/linux/ubuntu/scripts/act.sh. It is very similar to the image in this repo, except that it includes all of the apt packages in the official github runners and does not include:
liblttng-ust0orliblttng-ust1awsclipodmanbuildahskopeogh(github-cli)
This might make the most sense as the base image for our default image. However, they also have requests like catthehacker/docker_images#61 to add things like python3-pip to the act image. Clearly someone still needs to make some editorial decisions about what a reasonable base image includes.
Recommendation
Ultimately I'm not convinced that user story (3) will actually provide much value compared to (2). Supporting (1) is likely to come with a high maintenance or support burden and is likely to be overkill for most users.
For my use cases, the image today works great. I have my own Dockerfile that I use to add some additional python setup based on github's own setup script.
FROM myoung34/github-runner:ubuntu-focal
ENV AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
RUN apt-get update && \
cd /tmp && \
git clone https://github.com/actions/virtual-environments.git && \
echo '#!/usr/bin/env bash' > invoke_tests && \
chmod +x invoke_tests && \
cd virtual-environments/images/linux/scripts && \
PATH="/tmp:$PATH" HELPER_SCRIPTS=$PWD/helpers bash installers/python.sh && \
cd / && \
rm -rf /tmp/* && \
rm -rf /var/lib/apt/lists/* \
;
ENV PIPX_BIN_DIR=/opt/pipx_bin
ENV PIPX_HOME=/opt/pipx
RUN chown -R runner $PIPX_BIN_DIR $PIPX_HOME
ENV PATH="$PIPX_BIN_DIR:$PATH"The biggest issue I encounter today is the additional apt repos sometimes breaking and causing issues in my CI runs. My preference would be to keep the number of 3rd party apt repos to a minimum.
The best path forward is probably to either build on top of catthehacker/ubuntu:act-20.04 or adopt their method of installing all of the suggested apt packages. This will make our image larger, but will also make it easier for users to follow my example above and install toolchains using https://github.com/actions/virtual-environments. We can then install a curated set of packages like gh, awscli, etc.
Our images would then be:
basebuilt fromDockerfile.basethat includes:- the
aptpackages needed for running our setup scripts - any minimal editorializing of packages (
gh,awscli, etc.)
- the
mainimage that includes the actions runner and setup scriptsextraimage based onmainthat includes:pwsh(based on https://github.com/catthehacker/docker_images/blob/master/linux/ubuntu/scripts/pwsh.sh) which is needed for some of the official tooling- the setup needed to easily run additional toolchains and packages based on https://github.com/actions/runner-images/tree/main/images/linux/scripts/installers
Users would be expected to use the extra image as their base and then make their own image (this is unlikely to work in all cases for arm64).
FROM myoung34/github-runner:ubuntu-focal-extra
RUN /bin/bash /imagegeneration/installers/basic.sh \
&& /bin/bash /imagegeneration/installers/python.sh \
&& /bin/bash /imagegeneration/installers/firefox.sh \
&& /bin/bash /imagegeneration/installers/selenium.sh \
;
RUN . /etc/environment \
&& chown -R runner $PIPX_BIN_DIR $PIPX_HOME \
;I think the only modification to the existing image would be to source /etc/environment and re-export all of the environment variables in entrypoint.sh.
@myoung34 I think I've developed something similar to this, please check it out!
https://github.com/sansterbioanalytics/unified-actions-runner
Hey all, after thinking on it a bunch: I dont think this is something I personally want to maintain
That said, I think I'm happy to provide guidance etc, but I think the best option is for someone to automate a downstream "purpose built" image or images
I'm happy to help give guidance on automation, but ideally:
- A new repo(s) that triggers when I release a tag (which is triggered when actions/runner releases a tag)
- Does a FROM: myoung34/github-runner
- builds all the purpose built stuff
I say this because this repo is already a bit tricky as-is in someways but also I personally have 0 use case for this, so the idea of maintaining purpose built images for things I don't use or wish to troubleshoot is not appealing
I'm going to close this as "not possible" because the aim of this repo is to be inline with the actions/runner upstream capabilities for self-hosting or a platform for others to use as suggested here