The Python script converts YAML on the left to the Dockerfile on the right
macros: # Generated by https://github.com/larytet/dockerfile-generator on sub-2 172.20.20.180
get_release:
- cat /etc/*release # sudo docker build --tag centos7:latest --file $HOME/dockerfile-generator/Dockerfile.centos7 .
- gcc --version # sudo docker run --name centos7 --tty --interactive centos7:latest
# sudo docker start --interactive centos7
build_essential_centos: # Docker configuration:['{\n', '\t"dns": ["172.20.15.1", "8.8.8.8"],\n', '\t"experimental": true,\n', '\t"graph": "/home/user/docker"\n', '}\n', '\n']
- gcc
- gcc-c++
- make FROM centos:centos7
MAINTAINER automatically generated from /home/user/dockerfile-generator/containers.yml
make_dir:
- mkdir -p /etc/docker RUN set +x && `# Generate README file` && echo -e 'Generated by https://github.com/larytet/dockerfile-generator on sub-2 172.20.20.180\n\
Build the container. See https://docs.docker.com/engine/reference/commandline/build\n\
environment_vars: sudo docker build --tag centos7:latest --file $HOME/dockerfile-generator/Dockerfile.centos7 .\n\
# I need an env variable referencing a persistent folder Run the previously built container. See https://docs.docker.com/engine/reference/commandline/run\n\
- SHARED_FOLDER /etc/docker sudo docker run --name centos7 --network='host' --tty --interactive centos7:latest\n\
Start the previously run container\n\
containers: sudo docker start --interactive centos7\n\
Connect to a running container\n\
centos7: sudo docker exec --interactive --tty centos7 /bin/bash\n\
base: centos:centos7 Save the container for the deployment to another machine. Use 'docker load' to load saved containers\n\
packager: rpm sudo docker save centos7 -o centos7.tar\n\
install: Remove container to 'run' it again\n\
- $build_essential_centos sudo docker rm centos7\n\
- rpm-build ' > README
run: ENV SHARED_FOLDER /etc/docker
- $get_release
env: RUN `# Install packages` && set -x && \
- $environment_vars yum -y -v install gcc gcc-c++ make rpm-build && \
yum clean all && yum -y clean packages
RUN `# Execute commands` && set -x && \
`# $get_release` && \
cat /etc/*release && \
gcc --version
A Dockerfile is a list of commands which should be executed in the container. What if I want to create a few non trivial dockerfiles which share a significant amount of code. I introduce a YAML configuration file and a Python script which parses the configuration file. This is something like https://jsonnet.org/ but for a Dockefile
The original problew was to generate mutliple Dockerfiles with significant overlap. Specifically I needed (cross) build environments to produce kernel modules and STAP modules for different Linux kernels. My approach to the problem was a container wiht Linux header files, Linux kernel symbols, correct tool chain, dependencies. The approach required a tool which generated a Dockerfile on the fly given the kernel version and distribution.
The main goals:
- Keep multiple dockerfiles in a single place
- All dockerfiles have a consistent structure
- Opens an opportunity to implement template based Dockerfiles generation
- Enforce specific "best" practices like order of operations, running Dockerfile linter
- Simplify control number of layers
- Sort the lists of arguments
- Support macros, reduce code duplication
- Switch to a different OS/OS release/different version of a package is trivial thanks to macros
- Generate help, usage tips, run time and source code comments automatically
- Convenient support for generation of shell scripts, README files
- Generate multiple dockerfiles from a single YAML configuration file
- Specify the expected docker environment in the Dockerfile, for example version of the docker, daemon.json
- Can force order of the Dockerfile updates and reduce the image size
- what else?
Install missing Python packages
pip install -r requirements.txtInstall DockerCE. For example, something like this shall work:
# Run as root:
wget https://get.docker.com -O - | sh
systemctl stop docker
# Use overlayFS (because it is cool and recommended)
CONFIGURATION_FILE=$(systemctl show --property=FragmentPath docker | cut -f2 -d=)
cp $CONFIGURATION_FILE /etc/systemd/system/docker.service
perl -pi -e 's/^(ExecStart=.+)$/$1 -s overlay/' /etc/systemd/system/docker.service
systemctl daemon-reload
systemctl start dockerGenerate dockerfiles (modify file ./containers.yml if necessary)
rm -f Dockerfile.* ;./dockerfile-generator.py --config=./containers.yml Build Docker images. This operation should be done every time ./containers.yml is modified
for f in ./Dockerfile.*; do filename=`echo $f | sed -E 's/\..Dockerfile.(\S+)/\1/'`;echo Processing $filename;sudo docker build -t $filename -f $f .;sudo docker save $filename -o $filename.tar;doneCheck that the artefacts are created
ls -al Dockerfile* *.tarStop and remove:
#docker stop $(docker ps -a -q)
#docker rm $(docker ps -a -q)
#docker rmi $(docker images -q)
./remove-all.sh- https://github.com/avirshup/DockerMake
- http://docteurklein.github.io/2015/01/11/docker-auto-builds-and-me/
- https://blog.dockbit.com/templating-your-dockerfile-like-a-boss-2a84a67d28e9
- Support copy_secret which removes the files in the end of the build. The user is expected to "squash" the layer when running the build.
- The "generate" methods are messy, require refactoring
- Injection of comments is not consistent
- Better management of layers and caching
Add the following line In the /etc/default/docker
# Use DOCKER_OPTS to modify the daemon startup options.
DOCKER_OPTS="--dns 172.20.15.1 --dns 8.8.8.8 --dns 8.8.4.4 -g /home/USERNAMEHERE/docker"
Create file /etc/docker/daemon.json
{
"dns": ["172.20.15.1", "8.8.8.8"],
"experimental": true,
"graph": "/home/USERNAME/docker"
}