The MORPHUZZ (2022) paper found that QEMU was improperly saving / loading virtual device state with its snapshotting mechanism, which was leading to state leakage between snapshot loads. The goal of this project is to automate a method for saving, fuzzing, and analyzing virtual device state information to detect if state information is being improperly saved / loaded by different QEMU virtual devices.
More in-depth information on the motivation, steps taken, results, and documentation of this project can be found on this slide deck.
The code within this repo contains the scripts, function files, and altered QEMU source code for the E1000 (network device), AM53C974 (storage device), and CCID-Card-Passthru (smart card) devices that enables the virtual device state fuzzing and analysis setup. The virtual device code utilizes QEMU's VMState macros to save the raw state data for each field defined in the respective virtual device state data structure right before a snapshot is saved (in pre_save) and right after a snapshot is loaded (in post_load). These files include metadata that enable individual state fields to be compared to detect any differences between what each state field was when it was saved vs. when it was loaded by the snapshot (to detect any possible state leaks). The altered virtual device code also define a pre_load function to randomize virtual device state before loading the snapshot (to fuzz the state before the load and help detect possible sources of state leakage).
This code also includes a fuzzing script that automates saving, fuzzing, loading, and analyzing the saved vs. loaded states by connecting to QEMU Monitor (via socket), running savevm, then looping to loadvm and compare the saved vs. loaded state files generated by pre_save and post_load.
The project also includes a script dedicated to automatically generating the changes implemented for the 3 specified virtual devices here. This script isn't fully finished. It is currently able to scrape QEMU virtual device source code for all virtual device field information, with the next steps being to use the Jinja2 library to generate (or append to if they already exist) the pre_save, post_load, and pre_load functions made in qemu_source.
The following is instructions on how to replicate the setup used throughout the project.
Documentation on how to install QEMU dependencies needed for creating a QEMU VM can be found on the QEMU GitHub page and here
For setting up an Ubuntu VM, you will also need to install an Ubuntu iso file (throughout the project I used Ubuntu Desktop 20.04, but Tiny Core Linux might also work).
// Create a qcow2 QEMU image file
qemu-img create -f qcow2 test_vm.qcow2 16G
// Boot up VM with the installed iso file
qemu-system-x86_64 -m 1024 -enable-kvm \
-drive if=virtio,file=test_vm.qcow2,cache=none \
-cdrom ubuntu-20.04.6-desktop-amd64.iso
// Go through setup process for Ubuntu OS
Install QEMU source code and dependencies (instructions from QEMU docs)
git clone https://gitlab.com/qemu-project/qemu.git
cd qemu
git submodule init
git submodule update --recursive
Files to replace:
- /path/to/qemu/meson.build: qemu_source/meson.build
- /path/to/qemu/hw/net/e1000.c: qemu_source/e1000.c
- /path/to/qemu/hw/usb/ccid-card-passthru.c: qemu_source/ccid-card-passthru.c
- /path/to/qemu/hw/scsi/esp-pci.c: qemu_source/esp-pci.c
- Add fuzz_scripts/state-save.c and fuzz_scripts/state-save.h to /path/to/qemu/hw directory
These changes swap out the current source code for the 3 virtual devices with the updated version created that saves virtual device state information. Swapping out the meson.build file tells the build / configuration step to build QEMU with the functions created for saving and randomizing state field information.
// Build QEMU with the updated source files
cd /path/to/QEMU
./configure --enable-sdl --enable-smartcard --extra-cflags="-g" --target-list=x86_64-softmmu
make
// This will build the qemu-system-x86_64 executable with the modified files and functions
// (located in qemu/build/x86_64-softmmu directory)
The following are the commands I used when testing for running the QEMU VM and running the fuzz script
cd /path/to/qemu
// Compile fuzzing script to an executable
gcc /path/to/QEMU-statecompare/fuzz_scripts/trev_monitor.c -o fuzz_qemu
// Run QEMU with necessary attached devices and monitor on a socket
./build/x86_64-softmmu/qemu-system-x86_64 -hda ./trevor_test.img -m 6G -smp cores=6 -display sdl -chardev socket,server=on,host=0.0.0.0,port=2001,id=ccid,wait=off -usb -device usb-ccid -device ccid-card-passthru,chardev=ccid -device am53c974,id=scsi0 -monitor unix:/tmp/qemu-monitor.sock,server,nowait
// Run fuzz script (takes text file with list of devices to check comparison for)
./fuzz_qemu /path/to/QEMU-statecompare/fuzz_scripts/device_inputs.txt
// Output will be written to /path/to/qemu/fuzz_log.txt
If you're just getting started with QEMU, here are a few helpful links:
Links to documentation taken throughout the project: