MARSS-RISCV (Micro-ARchitectural System Simulator - RISCV) is a open source, cycle-accurate single core full-system (Linux) micro-architectural simulator for the RISC-V ISA built upon TinyEMU emulator (https://bellard.org/tinyemu) by Fabrice Bellard and uses its code for all the device emulation and configuration. It consists of detailed cycle accurate models of a modern RISC-V In-order and Out-of-order processor with branch prediction unit and a complete memory hierarchy. It is currently being developed and maintained by CAPS (Computer Architecture and Power Aware Systems Research Group) at the State University of New York at Binghamton. Being a true full system simulator, MARSS-RISCV can simulate all of the system in a cycle accurate fashion including OS code, libraries, interrupt handlers etc.
Currently, our simulator is in alpha status as we are validating the cycle accuracy using various development boards. The simulated in-order core is tested and operational, however, the simulated out-of-order core is in microarchitectural testing phase.
- Features
- Getting started with the simulator running a Linux guest
- Running full system simulations
- Viewing live simulation stats
- Generating simulation trace
- Future work
- Technical notes
- Authors
- Acknowledgment
- License
- True full system simulator: simulates all of the system in a cycle accurate fashion including the bootloader, OS code, libraries, interrupt handlers, user level applications etc
- Fully configurable, cycle-accurate, in-order and out-of-order single-core RISC-V CPU
- Multiple execution units with configurable latencies (execution units can be configured to be pipelined)
- 2-level cache hierarchy with various allocation and miss handling policies
- A simple DIMM based DRAM model that simulates row-buffer (open-page) hits
- Bi-modal and 2-level adaptive (Gshare, Gselect, GAg, GAp, PAg, PAp) branch prediction support
- Supports
RV32GCandRV64GC(user level ISA version2.2, privileged architecture version1.10) - VirtIO console, network, block device, input and 9P filesystem
- JSON configuration file
- Easy to install, use and modify
For internal micro-architectural details, see MARSS-RISCV Micro-architecture Documentation.
- 32-bit or 64-bit Linux machine
- Libcurl, OpenSSL and SDL Libraries
- Standard C compiler
Make sure that you have all the dependencies (ssl, sdl, and curl libraries) installed on the system. For Debian-based (including Ubuntu) systems, the packages are: build-essential, libssl-dev, libsdl1.2-dev, libcurl4-openssl-dev.
$ sudo apt-get update
$ sudo apt-get install build-essential
$ sudo apt-get install libssl-dev
$ sudo apt-get install libsdl1.2-dev
$ sudo apt-get install libcurl4-openssl-devFirst, clone the simulator repository:
$ git clone https://github.com/bucaps/marss-riscv
Then, cd into the simulator source directory:
$ cd marss-riscv/src/Modify CONFIG_XLEN and CONFIG_FLEN options in the Makefile as required. Supported XLEN values are 32 and 64. Supported FLEN values are 0, 32 and 64.
Then, compile the simulator using:
$ makeUsing pre-built bootloader, kernel and userland images is the easiest way to start. The pre-built images are located in the marss-riscv-images repository. To clone it, make sure you have Git LFS installed on your system, and type:
$ git clone https://github.com/bucaps/marss-riscv-imagesThe userland image needs to be decompressed before running the simulator:
$ cd marss-riscv-images/riscv32-unknown-linux-gnu/
$ xz -d -k -T 0 riscv32.img.xzWhen decompression finishes, launch the simulator with:
$ ../../marss-riscv riscvemu.cfgSimulation parameters can be configured using riscvemu.cfg, RISCVEMU JSON configuration file.
By default, the simulator will boot in "snapshot" mode, meaning it will not retain the file system changes after it is shut down. In order to persist the changes, pass -rw command line argument to the simulator. In that case, it may also be desirable to grow the userland image (has roughly 200MB of available free space by default). More information about how to grow it can be found here.
By default, guest boots in emulation mode. To start in simulation mode run with -simstart command line option.
Once the guest boots, we need to initialize the environment. Normally, this should happen automatically but due to an unresolved bug it needs to done explicitly:
# export PYTHONPATH=/usr/lib64/python2.7/site-packages/
# env-updateThe system is ready for use. It has a working GCC compiler, ssh, git, and more. It has a virtual network interface. However, there's no host-to-guest routing, so it's not possible to ssh into the guest.
By default, Ctrl-C will not kill the simulator. The command halt will cleanly shutdown the guest. Alternatively, you can pass the -ctrlc command line argument to the simulator, which will allow it to be killed using Ctrl-C.
Once you have access to the guest machine terminal, see next section for running simulations.
We have provided two special checkpoint instructions, SIM_START() and SIM_STOP(), which inform MARSS-RISCV to enable and disable simulation mode respectively, when they are encountered during instruction processing. When the simulation is disabled, MARSS-RISCV runs in the emulation mode. For this purpose, we have used two special unused registers from user mode CSR address space, 0x800 and 0x801. Using these two special markers, it is possible to simulate any section in the source code. However, after inserting these markers, the source code must be recompiled inside the guest OS using the installed gcc toolchain. Below code shows a simple hello world program with checkpoints. On encountering SIM_STOP() checkpoint during the simulation, all the performance stats are saved in the file sim_stats_file as configured in the simulator configuration file.
/* hello_world.c */
#include <stdio.h>
#define SIM_START() asm("csrs 0x800,zero")
#define SIM_STOP() asm("csrs 0x801,zero")
int main()
{
SIM_START();
printf("Hello World\n");
SIM_STOP();
return 0;
}Alternatively, you can use the provided simulation script (simulate.c) provided here which forks a child process. Child enters the simulation mode and execs the command. Parent process waits for the child to complete and then switches MARSS-RISCV back to emulation mode. Using this script it is possible to simulate any program without need to modify and re-compile the source code. Since child switches to simulation mode before calling exec(), exec() also runs in the simulation mode. Hence, performance statistics generated at the end of the simulation will also include stats for exec().
You can view live simulation stats using the provided stats-display tool. First, open a new terminal before executing the simulator and launch stats-display:
$ ./stats-displayThen launch the simulator on a different terminal with -stats-display command-line option.
To generate instruction commit trace of the programs running in the simulation mode, add CONFIG_SIM_TRACE flag to the CFLAGS variable in the makefile and recompile. Generated trace is saved in the file sim_trace_file as configured in the simulator configuration file.
Sample trace is shown below:
cycle = 112 pc = 0x6aaaadf8 insn = 0x85be863a c.mv a2,a4 mode = PRV_U status = simulated
cycle = 113 pc = 0x6aaaadfa insn = 0x250385be c.mv a1,a5 mode = PRV_U status = simulated
cycle = 115 pc = 0x6aaaadfc insn = 0xfa842503 lw a0,s0,-88 mode = PRV_U status = simulated
- Support for 3-stage in-order pipeline
- Support for return address stack
- Support for branch prediction unit, speculative execution and age-ordered instruction issue logic in the out-of-order core
- Cycle accuracy validation using various RISC-V development boards
- Integrating DRAM simulator (like DRAMSim2) with MARSS-RISCV, to get accurate memory stats
The floating point emulation is bit exact and supports all the specified instructions for 32 and 64 bit floating point numbers. It uses the new SoftFP library.
The standard HTIF console uses registers at variable addresses which are deduced by loading specific ELF symbols. TinyEMU does not rely on an ELF loader, so it is much simpler to use registers at fixed addresses (0x40008000). A small modification was made in the "riscv-pk" boot loader to support it. The HTIF console is only used to display boot messages and to power off the virtual system. The OS should use the VirtIO console.
The easiest way is to use the "user" mode network driver. No specific configuration is necessary.
MARSS-RISCV also supports a "tap" network driver to redirect the network traffic from a VirtIO network adapter.
You can look at the netinit.sh script to create the tap network interface and to redirect the virtual traffic to Internet thru a NAT. The exact configuration may depend on the Linux distribution and local firewall configuration.
The VM configuration file must include:
eth0: { driver: "tap", ifname: "tap0" }
and configure the network in the guest system with:
ifconfig eth0 192.168.3.2 route add -net 0.0.0.0 gw 192.168.3.1 eth0
MARSS-RISCV supports the VirtIO 9P filesystem to access local or remote filesystems. For remote filesystems, it does HTTP requests to download the files. The protocol is compatible with the vfsync utility. In the "mount" command, "/dev/rootN" must be used as device name where N is the index of the filesystem. When N=0 it is omitted.
The build_filelist tool builds the file list from a root directory. A simple web server is enough to serve the files.
The '.preload' file gives a list of files to preload when opening a given file.
MARSS-RISCV supports an HTTP block device. The disk image is split into small files. Use the 'splitimg' utility to generate images. The URL of the JSON blk.txt file must be provided as disk image filename.
- Copyright (c) 2017-2019 Gaurav Kothari {gkothar1@binghamton.edu}
- Copyright (c) 2018-2019 Parikshit Sarnaik {psarnai1@binghamton.edu}
This work was supported in part by DARPA through an award from the SSITH program. We would like to thank Gokturk Yuksek, Ravi Theja Gollapudi and Kanad Ghose for assistance with the internal details of TinyEMU and the development of the MARSS-RISCV.
This project is licensed under the MIT License - see the src/MIT-LICENSE.txt file for details.
The SLIRP library has its own license (two clause BSD license).