ixy-ci

A CI service to test the ixy userspace network driver and its derivatives. The basic idea is to spawn three independent VMs which take on the following roles:

pktgen pushes network packets (with sequence numbers) into network 1
fwd forwards all packets from network 1 to network 2
pcap captures packets from network 2

With this setup we can simultaneously test three applications and make sure that the whole scenario works correctly by inspecting the captured packets from pcap.

Currently only tests the Virtio driver. Testing the ixgbe driver is planned but requires infrastructure changes.

While ixy-ci does make sure that the build finishes correctly, you still may want to use ixy-ci in conjunction with traditional CI services to check builds across a larger variety of OS environments or to check things like formatting/linting.

What is currently being checked?

Correct amount of packets captured
Sanity check of the largest received sequence number (between n_pcap and 2 * n_pcap)
No duplicate packets received

How to test a new repository with ixy-ci

To use ixy-ci you only need to follow these instructions:

Create a GitHub webhook for your repository (in your repository settings)
- URL: https://ci.ixy.rs/github/webhook
- Content type: application/json
- Secret (e.g. openssl rand -base64 48); make sure to save this somewhere
- Events: Issue comments & Pushes
Securely send your webhook secret to your ixy-ci administrator
Open a PR adding a ixy-ci.toml to your repository (see ixy-ci.toml.example for reference)
Also make sure that your applications match the expected command line interface as described below
Comment @ixy-ci test in your PR until the tests pass :)

Required command line interface of applications

pktgen <pci addr>
fwd <pci addr src> <pci addr dst>
pcap <pci addr> <pcap output file> <stop after n packets>

The packets that are generated by pktgen are expected to follow the same structure and size as in the ixy pktgen example.

ixy-ci setup instructions

These instructions are only needed when you want to deploy your own instance of ixy-ci.

OpenStack

ixy-ci assumes a dedicated OpenStack project with no other users.
Network pktgen-fwd
- Port pktgen
- Port fwd-in
Network fwd-pcap
- Port fwd-out
- Port pcap
Networks require a subnet for port creation/attachment to succeed (disable DHCP)
- This may be incorrect since it seems to work without a subnet throught the openstack CLI
Disable "Port Security" on all ports
Create a keypair for ixy-ci to SSH into the spawned VMs

`config.toml`

See config.toml.example for the general format of the config.toml. The OpenStack login information can be extracted from a generated clouds.yaml. The only additional information you have to manually query is the project domain. You can do that like this:

# First retrieve the domain id
openstack --os-cloud openstack project show <project_name>
# Then retrieve the actual domain name
openstack --os-cloud openstack domain show <domain_id>

OpenStack CLI

Currently ixy-ci requires that the OpenStack CLI is available due to missing APIs in the openstack crate.

GitHub bot account

ixy-ci requires a GitHub account to post results and to interact with the GitHub API. Any account should work though we advise to use a dedicated bot account. You need to create a personal access token (GitHub / Setting / Developer settings) with access to the public_repo scope.

Deploy with Docker

make # Ctrl+C once running
docker build . -t ixy-ci
docker volume create ixy-ci-config
cp ~/.ssh/id_rsa /var/lib/docker/volumes/ixy-ci-config/_data/
cp config.toml.example /var/lib/docker/volumes/ixy-ci-config/_data/config.toml
docker run --mount source=ixy-ci-config,target=/config -p 127.0.0.1:9999:8080 --restart always -d --name ixy-ci ixy-ci

MicroStack

For local development you can use MicroStack which deploys OpenStack on your machine. (in a snap so without affecting the rest of your system too much)

Here are complete instructions for getting up and running:

# Setup MicroStack
sudo snap install microstack --devmode --beta
sudo microstack.init --auto --control
# Get password for "admin"; Web UI is at http://10.20.20.1
sudo snap get microstack config.credentials.keystone-password

# Import Debian 10 image
wget https://cdimage.debian.org/cdimage/openstack/current-10/debian-10-openstack-amd64.qcow2
microstack.openstack image create --container-format bare --disk-format qcow2 --file debian-10-openstack-amd64.qcow2 debian-10-openstack-amd64
rm debian-10-openstack-amd64.qcow2

# Create virtual networks
microstack.openstack network create pktgen-fwd --disable-port-security
microstack.openstack port create pktgen --network pktgen-fwd
microstack.openstack port create fwd-in --network pktgen-fwd

microstack.openstack network create fwd-pcap --disable-port-security
microstack.openstack port create fwd-out --network fwd-pcap
microstack.openstack port create pcap --network fwd-pcap

# Import SSH public key
microstack.openstack keypair create ixy-ci --public-key ~/.ssh/id_rsa.pub

# Adjust config.toml:
# Set the openstack password (see command above)
# Set the GitHub API token
# Adjust the path to the SSH private key

# After a reboot you can start MicroStack again with:
sudo snap start microstack

# Remove MicroStack again when you're done
sudo snap remove microstack --purge
# Restart computer to also get rid of left over virtual interfaces etc.

TODO

Only allow configured users to start tests (to prevent abuse)
Do more stuff concurrently once async/await is ready (also trussh instead of libssh2)
Fix issue where ixy-ci cannot be terminated via ctrl+c after a message has been posted on GitHub (related: graceful shutdown?)
Code documentation
Track down why OpenStack project domain is required (although OpenStack CLI doesn't need it)

Future feature plans

Test on master branch push (+ cronjob?) => endpoint for badges which redirect to shields.io
Dashboard with status about current job, queue, past results
Test ixgbe driver somehow (SR-IOV, pci passthrough?)
Integration with GitHub checks API

License

This project is licensed under the terms of the MIT license.

Bobo1239 / ixy-ci