mdw-octoml/microtvm-blogpost-eval

NOTE: Looking for the repo referenced from the original microTVM blog post? Check out the archived experimental-blogpost branch.

MicroTVM is an effort to run TVM on bare-metal microcontrollers. You can read more about the current design in the original Standalone microTVM Roadmap. This repo shows you how to run CIFAR10-CNN on the host machine and on an STM Nucleo-F746ZG development board.

Hardware you will need

A machine capable of running the microTVM Reference Virtual Machines, or a Linux machine with TVM and Zephyr installed.
STM Nucleo-F746ZG development board
- Autotuning can be sped up by adding more of these development boards.
micro USB cable

Software you will need

A computer capable of running a VM hypervisor (VirtualBox or Parallels).
Vagrant (you will install this in step 4).

Getting Started

Clone this repository (use git clone --recursive to clone submodules).
Clone the TVM repo: git clone --recursive https://github.com/apache/tvm tvm
Setup the microTVM Reference VM.
- NOTE: Use this vagrant up command instead of the one given there:
```
$ TVM_PROJECT_DIR=path/to/microtvm-blogpost-eval vagrant up --provider=<provider>
```
Choose virtualbox or parallels for <provider>, depending which VM hypervisor is installed. vmware support isn't available yet, but we're working on it.
Install extra dependencies. SSH to the VM, then, in the microtvm-blogpost-eval directory, run:
```
microtvm-blogpost-eval$ pip3 install -r requirements.txt
```
Setup PYTHONPATH (run from microtvm-blogpost-eval):
```
$ export PYTHONPATH=$(pwd)/python
```
Attach the USB device to the Reference VM.

Running host-driven

You can run the model in a host-driven configuration with the following command:

$ python -m micro_eval.bin.eval cifar10_cnn:micro_dev:data/cifar10-config-validate.json --validate-against=cifar10_cnn:interp:data/cifar10-config-validate.json

This command builds a Zephyr binary and flashes it to the device. It also builds a similar model to run on the host. It drives both models with the same inputs and displays the output in terminal.

Using Jupyter notebook

This process is captured in the Jupyter notebook in tutorial/standalone_utvm.ipynb. You can run this as follows:

(If running on the Reference VM) In the tvm repo, edit apps/microtvm/reference-vm/zephyr/Vagrantfile and add a line as follows:
```
config.vm.network "forwarded_port", guest: 8090, host: 8090
```

Bounce the VM:

$ vagrant halt
$ TVM_PROJECT_DIR=path/to/microtvm-blogpost-eval vagrant up

SSH to the VM. Be sure to reset the PYTHONPATH:

$ cd /path/to/microtvm-blogpost-eval
$ export PYTHONPATH=$(pwd)/python

Install jupyter: $ pip3 install jupyter
Launch the notebook. In this blogpost repo, run $ python -mjupyter notebook --no-browser --port 8090 --ip=0.0.0.0

Copy the 127.0.0.1 URL from your console to your browser. This should bring up Jupyter notebook--navigate to tutorial/standalone_utvm.ipynb and you should be set.

Running standalone

You can also run the Relay CIFAR10 model standalone on-device. First, translate the model into C in the standalone project:

$ python -m micro_eval.bin.standalone cifar10_cnn:micro_dev:data/cifar10-config-validate.json

Now, flash the project onto the device using Zephyr commands:

$ cd standalone
$ west build -b <your_board>
$ west flash

To view the model output, first run miniterm:

$ python -mserial.tools.miniterm /dev/ttyACM2 115200   # NOTE: replace ttyACM2 with your serial port
--- Miniterm on /dev/ttyACM2  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

Now, reset the board and you should see:

*** Booting Zephyr OS build zephyr-v2.4.0  ***
uTVM Standalone Demo
TVM complete! Output: (varies depending on the model spec, but should match)

Running autotuning

NOTE: This section is WIP with the new on-device runtime.

Debugging

There are a lot of moving pieces here and it's easy for the system to fail. Here I've tried to document some of the problems you can run into, and how to solve them.

Generally speaking, first try to enable debug logs with --log-level=DEBUG.

RPC server times out

You may see cases where the RPC server times out waiting for the session to be established. This can happen for several reasons:

A communication problem between the board and VM. Sometimes the serial port can drop bytes. Try unplugging the board and re-attaching it several times, which may clear it up.
A fault occurs on the device at startup. Try editing the script in question and set the DEBUG variable to True. Then, open another terminal and launch python -mtvm.exec.microtvm_debug_shell. This window will show the debugger while the script is running. Finally, re-run the edited script in your other terminal. You should see GDB appear in the microtvm_debug_shell window and the script will pause at a prompt to let you set up the debugger. To solve session establishment problems, try debugging the UART connection or the main() startup.

If the RPC server times out later on, it's likely there's been a fault in operator execution. Use the approach from point 2 above, and set a breakpoint at TVMFuncCall or the function in question.

mdw-octoml / microtvm-blogpost-eval