FIRMADYNE is an automated and scalable system for performing emulation and dynamic analysis of Linux-based embedded firmware. It includes the following components:
- modified kernels (MIPS: v2.6.32, ARM: v4.1, v3.10) for instrumentation of firmware execution;
- a userspace NVRAM library to emulate a hardware NVRAM peripheral;
- an extractor to extract a filesystem and kernel from downloaded firmware;
- a small console application to spawn an additional shell for debugging;
- and a scraper to download firmware from 42+ different vendors.
We have also written the following three basic automated analyses using the FIRMADYNE system.
- Accessible Webpages: This script iterates through each file within the filesystem of a firmware image that appears to be served by a webserver, and aggregates the results based on whether they appear to required authentication.
- SNMP Information: This script dumps the contents of the
public
andprivate
SNMP v2c communities to disk using no credentials. - Vulnerability Check: This script tests for the presence of 74 vulnerabilities using exploits from Metasploit and other sources.
In our 2016 Network and Distributed System Security Symposium (NDSS) paper, titled Towards Automated Dynamic Analysis for Linux-based Embedded Firmware, we evaluated the FIRMADYNE system over a dataset of 23,035 firmware images, of which we were able to extract 9,486. Using 60 exploits from the Metasploit Framework, and 14 previously-unknown vulnerabilities that we discovered, we showed that 846 out of 1,971 (43%) firmware images were vulnerable to at least one exploit, which we estimate to affect 89+ different products. For more details, refer to our paper linked above.
Note: This project is a research tool, and is currently not production ready. In particular, some components are quite immature and rough. We suggest running the system within a virtual machine. No support is offered, but pull requests are greatly appreciated, whether for documentation, tests, or code!
The following has been tested on a Ubuntu 14.04 machine. Other Debian-based systems should also be compatible.
First, clone this repository recursively and install its dependencies.
sudo apt-get install busybox fakeroot git kpartx netcat-openbsd nmap python-psycopg2 python3-psycopg2 snmp uml-utilities util-linux vlan
git clone --recursive https://github.com/firmadyne/firmadyne.git
The extractor depends on the binwalk tool, so we need to install that and its dependencies.
git clone https://github.com/devttys0/binwalk.git
sudo ./binwalk/deps.sh
sudo python ./binwalk/setup.py install
- For Python 2.x,
sudo apt-get install python-lzma
sudo -H pip install git+https://github.com/ahupp/python-magic
- Instead of upstream jefferson, it is recommended to install our jefferson fork, which supports extraction of additional file and compression types.
- Optionally, instead of upstream sasquatch, our sasquatch fork can be used to prevent false positives by making errors fatal.
Next, install, set up, and configure the database.
sudo apt-get install postgresql
sudo -u postgres createuser -P firmadyne
, with passwordfirmadyne
sudo -u postgres createdb -O firmadyne firmware
sudo -u postgres psql -d firmware < ./firmadyne/database/schema
To download our pre-built binaries for all components, run the following script:
cd ./firmadyne; ./download.sh
Alternatively, refer to the instructions below to compile from source.
To use QEMU provided by your distribution:
sudo apt-get install qemu-system-arm qemu-system-mips qemu-utils
Alternatively, use our modified version
of qemu-linaro for certain
firmware with an alphafs
webserver that assumes a fixed memory mapping (not
recommended), or upstream qemu.
- Set
FIRMWARE_DIR
infirmadyne.config
to point to the root of this repository. - Download a firmware image, e.g. v2.0.3 for Netgear WNAP320.
wget http://www.downloads.netgear.com/files/GDC/WNAP320/WNAP320%20Firmware%20Version%202.0.3.zip
- Use the extractor to recover only the filesystem, no kernel (
-nk
), no parallel operation (-np
), populating theimage
table in the SQL server at127.0.0.1
(-sql
) with theNetgear
brand (-b
), and storing the tarball inimages
../sources/extractor/extractor.py -b Netgear -sql 127.0.0.1 -np -nk "WNAP320 Firmware Version 2.0.3.zip" images
- Identify the architecture of firmware
1
and store the result in theimage
table of the database../scripts/getArch.sh ./images/1.tar.gz
- Load the contents of the filesystem for firmware
1
into the database, populating theobject
andobject_to_image
tables../scripts/tar2db.py -i 1 -f ./images/1.tar.gz
- Create the QEMU disk image for firmware
1
.sudo ./scripts/makeImage.sh 1
- Infer the network configuration for firmware
1
. Kernel messages are logged to./scratch/1/qemu.initial.serial.log
../scripts/inferNetwork.sh 1
- Emulate firmware
1
with the inferred network configuration. This will modify the configuration of the host system by creating a TAP device and adding a route../scratch/1/run.sh
- The system should be available over the network, and is ready for analysis. Kernel messages are logged to
./scratch/1/qemu.final.serial.log
../analyses/snmpwalk.sh 192.168.0.100
./analyses/webAccess.py 1 192.168.0.100 log.txt
mkdir exploits; ./analyses/runExploits.py -t 192.168.0.100 -o exploits/exploit -e x
(requires Metasploit Framework)sudo nmap -O -sV 192.168.0.100
- To access a console in the firmware, use a presupplied debug run script to access the default console (no network access), modify the network-enabled
run.sh
script to provide console access, or use the second console provided by the framework.
./scripts/run-debug.sh 1
nc -U /tmp/qemu.1.S1
- The following scripts can be used to mount/unmount the filesystem of firmware
1
. Ensure that the emulated firmware is not running, and remember to unmount before performing any other operations.
sudo ./scripts/mount.sh 1
sudo ./scripts/umount.sh 1
If you would like to compile the entire FIRMADYNE system from scratch without using our pre-built binaries, please follow the steps below.
In order to build any of the binaries used by FIRMADYNE, you will need three cross-compilation toolchains for the following architecture triples. Use only musl libc as the C runtime library for the toolchain; others have not been tested.
- arm-linux-musleabi
- mipseb-linux-musl
- mipsel-linux-musl
To simplify the process of building cross-compilation toolchains with musl, we recommend using the musl-cross project. Follow the below steps to build these toolchains from source, or alternatively click here to download our pre-built toolchains.
-
git clone https://github.com/GregorR/musl-cross.git
-
Modify or set the following variables in
defs.sh
BINUTILS_URL=http://ftp.gnu.org/gnu/binutils/binutils-2.25.1.tar.bz2
GCC_VERSION=5.3.0
GMP_VERSION=6.0.0a
MPC_VERSION=1.0.2
MPFR_VERSION=3.1.3
LIBELF_VERSION=master
MUSL_DEFAULT_VERSION=1.1.12
MUSL_GIT_VERSION=615629bd6fcd6ddb69ad762e679f088c7bd878e2
LANG_CXX=no
-
Modify or set the following variables in
config.sh
CFLAGS="-fPIC"
-
For little-endian MIPS, perform the following:
- set
TRIPLE=mipsel-linux-musl
inconfig.sh
- set
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.70.tar.xz
indefs.sh
- run
./clean.sh
to clean out any previous builds - run
./build.sh
to build and install the toolchain into/opt/cross
- set
-
For big-endian MIPS, perform the following:
- set
TRIPLE=mipseb-linux-musl
inconfig.sh
- set
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.70.tar.xz
indefs.sh
- run
./clean.sh
to clean out any previous builds - run
./build.sh
to build and install the toolchain into/opt/cross
- set
-
For little-endian ARM, perform the following:
- set
TRIPLE=arm-linux-musleabi
,GCC_BOOTSTRAP_CONFFLAGS="--with-arch=armv6 --with-float=softfp"
, andGCC_CONFFLAGS="--with-arch=armv6 --with-float=softfp"
inconfig.sh
- set
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v4.x/linux-4.1.17.tar.xz
indefs.sh
- run
./clean.sh
to clean out any previous builds - run
./build.sh
to build and install the toolchain into/opt/cross
- set
-
You should have the following directories, or wherever you installed the toolchains:
/opt/cross/arm-linux-musleabi
/opt/cross/mipseb-linux-musl
/opt/cross/mipsel-linux-musl
cd ./firmadyne/sources/console
make clean && CC=/opt/cross/arm-linux-musleabi/bin/arm-linux-musleabi-gcc make && mv console ../../binaries/console.armel
make clean && CC=/opt/cross/mipseb-linux-musl/bin/mipseb-linux-musl-gcc make && mv console ../../binaries/console.mipseb
make clean && CC=/opt/cross/mipsel-linux-musl/bin/mipsel-linux-musl-gcc make && mv console ../../binaries/console.mipsel
cd ./firmadyne/sources/libnvram
make clean && CC=/opt/cross/arm-linux-musleabi/bin/arm-linux-musleabi-gcc make && mv libnvram.so ../../binaries/libnvram.so.armel
make clean && CC=/opt/cross/mipseb-linux-musl/bin/mipseb-linux-musl-gcc make && mv libnvram.so ../../binaries/libnvram.so.mipseb
make clean && CC=/opt/cross/mipsel-linux-musl/bin/mipsel-linux-musl-gcc make && mv libnvram.so ../../binaries/libnvram.so.mipsel
git clone https://github.com/firmadyne/kernel-v4.1.git && cd kernel-v4.1
mkdir -p build/armel
cp config.armel build/armel/.config
make ARCH=arm CROSS_COMPILE=/opt/cross/arm-linux-musleabi/bin/arm-linux-musleabi- O=./build/armel zImage -j8
cp build/armel/arch/arm/boot/zImage ../firmadyne/binaries/zImage.armel
-
git clone https://github.com/firmadyne/kernel-v2.6.32.git && cd kernel-v2.6.32
-
For big-endian MIPS, perform the following:
mkdir -p build/mipseb
cp config.mipseb build/mipseb/.config
make ARCH=mips CROSS_COMPILE=/opt/cross/mipseb-linux-musl/bin/mipseb-linux-musl- O=./build/mipseb -j8
cp build/mipseb/vmlinux ../firmadyne/binaries/vmlinux.mipseb
-
For little-endian MIPS, perform the following:
mkdir -p build/mipsel
cp config.mipsel build/mipsel/.config
make ARCH=mips CROSS_COMPILE=/opt/cross/mipsel-linux-musl/bin/mipsel-linux-musl- O=./build/mipsel -j8
cp build/mipsel/vmlinux ../firmadyne/binaries/vmlinux.mipsel
During development, the database was stored on a PostgreSQL server.
Although we cannot redistribute binary firmware, the data used for our experiments is available here.
Below are descriptions of tables in the schema.
brand
: Stores brand names for each vendor.
Column | Description |
---|---|
id | Primary key |
name | Brand name |
image
: Stores information about each firmware image.
Column | Description |
---|---|
id | Primary key |
filename | File name |
brand_id | Foreign key to brand |
hash | MD5 |
rootfs_extracted | Whether the primary filesystem was extracted |
kernel_extracted | Whether the kernel was extracted |
arch | Hardware architecture |
kernel_version | Version of the extracted kernel |
object
: Stores information about each file in a filesystem.
Column | Description |
---|---|
id | Primary key |
hash | MD5 |
object_to_image
: Maps unique files to their firmware images.
Column | Description |
---|---|
id | Primary key |
oid | Foreign key to object |
iid | Foreign key to image |
filename | Full path to the file |
regular_file | Whether the file is regular |
permissions | File permissions in octal |
uid | Owner's user ID |
gid | Group's group ID |
product
Column | Description |
---|---|
id | Primary key |
iid | Foreign key to image |
url | Download URL |
mib_filename | Filename of the SNMP MIB |
mib_hash | MD5 of the SNP MIB |
mib_url | Download URL of the SNMP MIB |
sdk_filename | Filename of the source SDK |
sdk_hash | MD5 of the source SDK |
sdk_url | Download URL of the source SDK |
product | Product name |
version | Version string |
build | Build string |
date | Release date |
The results discussed in our paper were produced using pre-release versions of the following:
- toolchains:
BINUTILS_URL=http://ftp.gnu.org/gnu/binutils/binutils-2.25.1.tar.bz2
,GCC_VERSION=4.9.3
,GMP_VERSION=6.0.0a
,MPC_VERSION=1.0.2
,MPFR_VERSION=3.1.3
,LIBELF_VERSION=71bf774909fd654d8167a475333fa8f37fbbcb5d
,MUSL_DEFAULT_VERSION=1.1.10
,MUSL_GIT_VERSION=996d148bf14b477b07fa3691bffeb930c67b2b62
,LANG_CXX=no
- ARM:
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v3.x/linux-3.10.84.tar.xz
- MIPS:
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.67.tar.xz
- kernels:
- ARM: firmadyne-v3.10.92
- MIPS: firmadyne-v2.6.32.68 without
e2b9f315547ea50a65baad4899a4780078ab273e
and26bb3636c987fc7e145af73ddea6c10fa93bdae9
- console:
c36ae8553fa4e9c82e8a65752906641d81c2360c
- extractor:
5520c64bfa8554c5c17ab671aaed0fdeec91bf19
- libnvram:
b60e7d4d576b39dd46107058adb635d43e80e00d
- qemu-linaro:
4753f5e8126a00cc0a8559bfd9b47d6340903323
- binwalk:
f2ce2992695fae5477c46980148c89e9c91a5cce
- jefferson:
090a33be0be4aac8eee8d825447c0eb18dc8b51a
- sasquatch:
287e4a8e059d3ee7a5f643211fcf00c292cd6f4d
- jefferson: