Moirai7 / p2im-unit_tests

Unit test firmware evaluated in P2IM paper

This is the git repo for unit tests of P²IM paper.

For each unit test case, we include its source code, binary (ELF), and the processor-peripheral model instantiated by P2IM.

Disclaimer: only the following OS+MCU combinations are tested: (Arduino, STM32 F103RB), (Arduino, Atmel SAM3X8E), (NUTTX, STM32 F103RB), (RIOT, STM32 F103RB), (RIOT, Atmel SAM3X8E), (RIOT, NXP MK64FN1M0VLL12). There is no guarantee the test cases will function correctly on other OS+MCU combinations.

How is the git repo organized

Unit test cases are organized by OS and peripherals. For example, Arduino/ADC/ directory has

• Source code of Arduino ADC test case (.ino file),
• Firmware binary of Arduino ADC test case on f103 and sam3 MCU (.elf file),
• Test results (.csv file). Test results include all registers that are accessed by the unit test firmware, their category in ground truth (Reg cat column) and their category assigned by P2IM (Model Cat column). Note that when calculating register categorization accuracy in P2IM paper, only registers that have been read by the firmware are considered. This is because registers that are never read by the firmware do not influence firmware execution in P2IM at all. It does not matter whether those registers are correctly categorized or not.

How to instantiate model for the firmware and calculate statistics

We take Arduino ADC peripheral on SAM3 MCU as an example.

# modify the mi_path, qemu_path, objdump_path, model_stat_path, gt_path in <repo_path>/run.py

# cd to the directory where firmware is located
cd Arduino/ADC

# run.py instantiates processor-peripheral interface model, and calculates statistics
../../run.py sam3 SAM3X8EArduinoADC.elf # ../../run.py shows usage of the script

# peripheral_model-sam3.json is the peripheral model instantiated
# statistics are printed to stdout and sam3.csv

OS and test case specific README

Some steps, such as firmware compilation, are OS specific. Therefore, we have OS-specific README for Arduino, RIOT, and Nuttx.

Note that you need GNU Arm Embedded Toolchain to compile the firmware. Steps to set up the toolchain:

1. Download the toolchain from here.
2. Untar the downloaded file by tar xjf *.tar.bz2.
3. Add bin/ directory extracted into your $PATH environment variable.
4. Test if the toolchain is added to $PATH successfully by which arm-none-eabi-gcc.

Besides, there are test case specific README, such as RIOT/TIMER/README.md. Please don't forget to check them out.

Errata

credit: Marius Muench ([e-mail]), Wei Zhou ([e-mail]), and Tobias Scharnowski ([e-mail])

This test suite includes 66 valid unit tests, instead of 70 (the number we reported in p2im paper). This is because we found 4 more test cases not supported by the OS for some MCU, which are:

• NUTTX/I2C/NUTTX-F103-I2C.elf. Write function is not implemented by the OS.
• NUTTX/SPI/F103-NUTTX-SPI.elf. Write function is not implemented by the OS.
• Arduino/I2C/F103-RIOT-I2C.elf. "I2C_ADDR10" flag is not supported by Arduino on F103 (2 test cases for read and write respectively)

Citing our paper

@inproceedings {p2im,
title = {P2IM: Scalable and Hardware-independent Firmware Testing via Automatic Peripheral Interface Modeling},
author={Feng, Bo and Mera, Alejandro and Lu, Long},
booktitle = {29th {USENIX} Security Symposium ({USENIX} Security 20)},
year = {2020},
url = {https://www.usenix.org/conference/usenixsecurity20/presentation/feng},
}