stormwatch / Diary-of-a-Vagrant

Experiments on virtualization, learning Vagrant & Friends

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Diary of a Vagrant

I’ll try to install Vagrant and most provider backendslike VirtualBox, libvirt, VMware, etc. Then I’ll download some boxes to test, maybe also bechnmark and learn about them. My current computer runs Fedora 33, so many of the steps I’ll detail will apply to similar distributions without changes.

1 Hardware requirements

1.1 DONE Enable virtualization features in the BIOS setup

1.2 DONE Check if your system supports virtualization

If you have libvirt-client or lscpu already installed, you can run virt-host-validate 1 or LANG= lscpu | grep Virtualization respectively. Otherwise run
egrep "svm|vmx" /proc/cpuinfo

2 Virtual Machines providers

Due to time constrainsts, I won’t be able to make an exhaustive test of the different products right away. Some of these technologies are also called hypervisors which is a funny word that appeared in OS research papers around the sixties and seventies and denotes —at least in its origins— a supervisor of supervisors.

2.1 DONE Install VirtualBox

I enabled RPM Fusion and followed the instructions in RPM Fusion’s VirtualBox Howto. vboxdrv might fail to load because the module isn’t signed. I can either disable the secure boot option in the BIOS or sign the kernel modules. See also this bug report.

According to Vagrant’s Installation instructions VirtualBox might also fail if it is run alongside kvm. They suggest blacklisting the offending module.

2.1.1 DONE Install the guest additions

This package contains the VirtualBox Guest Additions

which support better integration of VirtualBox guests with the Host, including file sharing, clipboard sharing and Seamless mode,

Whatever that means.

2.2 DONE Install libvirt, qemu-kvm & utilities

sudo dnf -yq group install --with-optional virtualization

2.2.1 DONE Switch to modular dæmons

libvirtd is migrating from a monolithic to a modular dæmon architecture

The libvirt daemons, whether monolithic or modular, can often operate in two modes

System mode
the dæmon is running as the root user account, enabling access to its full range of functionality. A read-write connection to daemons in system mode typically implies privileges equivalent to having a root shell. Suitable authentication mechanisms must be enabled to secure it against untrustworthy clients/users.
Session mode
the dæmon is running as any non-root user account, providing access to a more restricted range of functionality. Only client apps/users running under the same UID are permitted to connect, thus a connection does not imply any elevation of privileges. DONE Stop the current monolithic daemon and its socket units

systemctl disable --now libvirtd.service
systemctl disable --now libvirtd{,-ro,-admin,-tcp,-tls}.socket

For stronger protection it is valid to use mask instead of disable too. DONE Enable the new daemons for the particular virtualizationd driver desired, and any of the secondary drivers to accompany it.

The following example enables the qemu and virtbvox drivers and all the secondary drivers:
for drv in qemu virtvbox interface network nodedev nwfilter secret storage
    systemctl unmask virt${drv}d.service
    systemctl unmask virt${drv}d{,-ro,-admin}.socket
    systemctl enable virt${drv}d.service
    systemctl enable virt${drv}d{,-ro,-admin}.socket

Start the sockets for the same set of daemons. There is no need to start the services as they will get started when the first socket connection is established.

for drv in qemu virtbvox network nodedev nwfilter secret storage
    systemctl start virt${drv}d{,-ro,-admin}.socket

If connections from remote hosts need to be supported the proxy daemon must be enabled and started

systemctl unmask virtproxyd.service
systemctl unmask virtproxyd{,-ro,-admin}.socket
systemctl enable virtproxyd.service
systemctl enable virtproxyd{,-ro,-admin}.socket
systemctl start virtproxyd{,-ro,-admin}.socket

The UNIX sockets allow for remote access using SSH tunneling. If libvirtd had TCP or TLS sockets configured, those should be started too

systemctl unmask virtproxyd-tls.socket
systemctl enable virtproxyd-tls.socket
systemctl start virtproxyd-tls.socket

2.2.2 TODO Configure the logging dæmon

for drv in log
    systemctl enable virt${drv}d.service
    systemctl enable --now virt${drv}d{,-admin}.socket

2.3 TODO Install VMware

3 DONE Install Vagrant

Vagrants abstracts different virtual machine providers under a common API and configuration layer. VirtualBox, Hyper-V, and Docker are supported right out of the box, for the rest I have to install plugins.

Here is an overview of the differnet installation methods that I tried out:

Fedora’s package (recommended)
Easiest installation. Lagging somewhat behind the upstream version. Won’t work with Windows guests. Moreover, asdf-ruby installed runtimes will conflict with Vagrant. The solution is to type asdf local ruby system inside the directory with the Vagrantfile, otherwise Vagrant will complain about not being able to find some ruby gems and not being able to require nokogiri. It has one inconvinience though in that Emacs won’t be able to find the binary for rubocop (this may affect solargraph too). I thought asdf local ruby system 2.7.2 would solve the issuse but it didn’t.2 In order to install vagrant-libvirt with vagrant plugin install vagrant-libvirt you have to run dnf install ruby-devel and asdf global ruby system before; otherwise you can install the vagrant-libvirt packaged by Fedora.
Easy installation. In fish shell just asdf plugin add vagrant; then asdf install vagrant latest; then asdf global vagrant x.y.z . As of [2020-12-04 vie]— vagrant plugin install vagrant-libvirt will fail so vagrant-libvirt kindly provides an alternative Docker based installation.
Download the official Vagrant package
Beware that the Upstream Vagrant isn’t compatible with the Fedora version of OpenSSL. Ruby in Vagrant is built against a bundled version of OpenSSL while during vagrant-libvirt installation, the system version of OpenSSL is used and later they clash. The workaround is to download OpenSSL and Kerberos 5 source RPM libraries, rebuild them and manually copying the shared libraries to /opt/vagrant/embedded/lib64. Then I can run vagrant plugin install vagrant-libvirt but make sure to remove the ~/.vagrant.d remnant from previous attempts, beforehand. I found the workarounds and relevant information for Fedora, RHEL 8, and CentOS 8 in vagrant-libvirt installation instructions and in this Vagrant issue. Similar problems also affect other software like Matlab and Scilab. I quote the following comment from another bug report:

…this means those products are using most probably outdated libraries w/o getting CVE bugfixes when the system gets them. I would open a bug report upstream to stop doing this stupid library interposing on all systems and do it only where the proper library version is missing (arguably they do this to handle RHEL/CentOS 6 which are stuck on openssl 1.0.2). That said at least Matlab is a proprietary product so… good luck, any number of things can break when they play fast and loose with critical libraries like openssl.

Install from source
clone the GitHub repo and bundle install. Then try to use the Docker image for vagrant-libvirt.
Use Windows
I followed Vagrant and Windows Subsystem for Linux. Looks promising but I remember having issues with. The preliminary tests I made with the native version and VirtualBox run fine.

3.1 Vagrant providers

A list of Vagrant providers is available in the Vagrant’s box search page:
  • aws
  • cloudstack
  • digitalocean
  • docker
  • google
  • hyperv
  • libvirt
  • lxc
  • openstack
  • parallels
  • qemu
  • rackspace
  • softlayer
Container-like virtualization for MacOS
  • virtualbox
  • vmware
  • vmware_desktop
  • vmware_fusion
  • vmware_ovf
  • wmware_workstation
  • vspehere
  • xenserver

Vagrant ships out of the box with support for VirtualBox, Hyper-V, and Docker.For the time being I’ll focus on VirtualBox, libvirt and maybe some of the cloud providers. VMware support on Vagrant isn’t free, and is advertised as being more performant and stable.

Vagrant is free and open source. While the VMware providers are not, the revenue is used to continue to develop, support, and grow Vagrant and the community around it.

3.2 Plugins

The core plugins implement the basic commands, providers, provisioners, hosts and guest functionalities.

Plugins are powerful, first-class citizens that extend Vagrant using a well-documented, stable API that can withstand major version upgrades.

In fact, most of the core of Vagrant is implemented using plugins. Since Vagrant dogfoods its own plugin API, you can be confident that the interface is stable and well supported.

Third party plugins implement additional providers, provisioners, and features like caching, networking, file sharing, etc. Here are some good starting points to look for Vagrant plugins:

Your distribution
if your distribution packages a vagrant plugin, it is a good indication that is widely used.
Avaliable Vagrant Plugins
in Vagrant’s wiki page at GitHub.
Notable plugins
is a section in the Awesome Vagrant page at GitHub.

There are currently src_shell{gem search –quiet –no-verbose –no-versions –no-details “(vagrant-|-vagrant)” | wc -l} packages containing vagrant. To retrieve a list with a short description of each one run:

gem search --quiet --no-verbose --no-versions --details "vagrant-|-vagrant"

3.2.1 Currently installed plugins

To get a list of currently installed plugins run:
vagrant plugin list
  • vagrant-cachier (1.2.1, global)
  • vagrant-libvirt (0.3.0, global) vagrant-cachier

Is best described by the author:

A Vagrant plugin that helps you reduce the amount of coffee you drink while waiting for boxes to be provisioned by sharing a common package cache among similiar VM instances.

4 TODO Install VMware

5 Interesting Vagrant boxes

5.1 Ailispaw’s Barge

a lightweight Linux distribution built with Buildroot specifically to run Docker containers.

5.1.1 Features

  • Built with Buildroot 2019.05 with Linux kernel v4.14.125 and glibc.
  • Docker v1.10.3 (~ latest version)
  • Support NFS synced folder
  • Support VirtualBox Shared Folder (VirtualBox only) with VirtualBox Guest Addition v6.0.4
  • Support Docker provisioner
  • Disable TLS of Docker for simplicity
  • Expose and forward the official IANA registered Docker port 2375
  • dumb-init binary is built-in /usr/bin
  • pkg command is built-in. You can install individual packages from Buildroot.
  • Enable to switch between Docker versions
  • 40 GB persistent disk
  • 13.8 MB

5.2 Bento

Boxes built using templates from the Chef’s Bento project. Currently they list:
  • FreeBSD
  • Debian
  • Oracle
  • Scientific
  • openSUSE
  • CentOS
  • Fedora
  • Ubuntu
  • Amazon Linux

5.3 Fedora

Official images mirrored in Fedora cloud base images for Vagrant

5.4 CentOS

Official images for CentOS.Besides the main images, they offer a proof of concept atomic host box that uses the aws backend.

5.5 FreeBSD

They claim to be “The FreeBSD Release Engineering Team” and I have no reason to doubt their sincerity.

5.6 alpine Linux

Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busybox.

5.7 Roboxes

Base Boxes Built by Benevolent Robotic Dogs.

Mirrored by generic. Provides:

  • Oracle
  • Fedora
  • Ubuntu
  • Devuan
  • Alpine
  • netBSD
  • openSUSE
  • Gentoo
  • HardenedBSD
  • RHEL 👀
  • DragonFlyBSD

6 TODO Testing Vagrant providers and boxes

Current status after running vagrant up
VirtualBoxlibvirt (qemu-kvm)VMware

7 TODO Selinux

Maybe this is needed for some boxes/providers.
setsebool -P nis_enabled 1

8 Further reading

Getting started with virtualization
is an excellent overview to using the virtualization capabilities in Fedora.
The Vagrant page
in the Fedora Wiki provides a useful Vagrantfile example to use the official Fedora cloud base images for Vagrant.
How to debug Virtualization problems
in the Fedora Wiki.
The Vagrant page at fedora DEVELOPER
Includes many configuration tips, especially for setting NFS shares up.
Installing and running Vagrant using qemu-kvm
Fedora Magazine, [2020-09-21].
Vagrant beyond the basics
Fedora Magazine, [2020-12-02].
How to run virtual machines with virt-manager
Fedora Magazine, [2019-07-22] .
Using Ansible to Provision Vagrant Boxes
Fedora Magazine, [2016-09-14].
How to Install Fedora as a VirtualBox guest
Fedora Magazine, [2017-03-20].
Getting started with Fedora CoreOS
Fedora Magazine, [2020-11-27].
is an open-source distributed virtualization solution, designed to manage your entire enterprise infrastructure. oVirt uses the trusted KVM hypervisor and is built upon several other community projects, including libvirt, Gluster, PatternFly, and Ansible.

9 Footnotes

1 As of [2021-01-04] virt-host-validate will yeld some warnings when using cgroups v2. This is already fixed.

2 For the time being I added the following to my init file:

(use-package rubocop
 :defer t
 :custom (rubocop-check-command
           (substring (shell-command-to-string "asdf which rubocop") 0 -1)
           " --format emacs")))


Experiments on virtualization, learning Vagrant & Friends

License:MIT License