Vmck

Vmck is a virtual machine service that runs QEMU and Docker images in a Nomad cluster.

It can be used as a provider for Vagrant:

vagrant plugin install vagrant-vmck
curl -OL https://github.com/mgax/vagrant-vmck/raw/master/example_box/Vagrantfile
vagrant up
vagrant ssh

And it has its own CLI:

pipenv run ./manage.py createjob
pipenv run ./manage.py ssh 1
pipenv run ./manage.py kill 1

Installation

Using docker

docker run --detach --restart always \
  --name cluster \
  --volume /opt/cluster/var:/opt/cluster/var \
  --volume /opt/vmck/var:/opt/vmck/var \
  --volume /var/run/docker.sock:/var/run/docker.sock:ro \
  --privileged \
  --net host \
  --env NOMAD_CLIENT_INTERFACE=wg0 \
  --env HOSTNAME=127.0.0.1 \
  --env SECRET_KEY=foo \
  mgax/vmck

Then go to consul (http://localhost:8500/ui/dc1/services) and wait for the health check lights to turn green.

Running locally

You still need a working Consul + Vault + Nomad cluster, maybe [liquidinvestigations/cluster][] can help you.

Create a file named .env with your local configuration:

SECRET_KEY=changeme
# DEBUG=true

Then install dependencies and run migrations:

pipenv install
pipenv run ./manage.py migrate

Then you can run the server:

pipenv run ./manage.py runserver

VM images

Each job spins up a QEMU virtual machine to evaluate the submission. It needs a disk image, which can be downloaded from https://github.com/mgax/vmck-images, or built using contrib/build.py, which downloads an Ubuntu cloud image, prepares a cloud-init configuration, applies it, and exports the resulting image. Run it with an argument to specify the output path:

./contrib/build.py /tmp/bionic.qcow

Testing

With a Nomad cluster running on localhost:4646, run pipenv run pytest, and enjoy. By default, the test suite will use a mock vm, implemented with a docker backend, instead of qemu. To run the tests with the qemu backend, set TESTING_BACKEND=qemu in the .env file.

To make the qemu tests run faster you can mirror the VM image locally and override the URL in the local .env file - see testsuite/settings.py for the default image.

Troubleshooting

QEMU fails to start with error qemu-system-x86_64: Invalid host forwarding rule 'tcp:${attr.unique.network.ip-address}:10674-:22' (Bad host address): the host address has changed (e.g. because it moved to a different WiFi hotspot). Restart Nomad and it should pick up the new address.

gabriel-v / vmck