NOTE: Looking for the repo referenced from the original microTVM blog post? Check out the archived experimental-blogpost
branch.
MicroTVM is an effort to run TVM on bare-metal microcontrollers. You can read more about the current design in the original Standalone microTVM Roadmap. This repo shows you how to run CIFAR10-CNN on the host machine and on an STM Nucleo-F746ZG development board.
- A machine capable of running the microTVM Reference Virtual Machines, or a Linux machine with TVM and Zephyr installed.
- STM Nucleo-F746ZG development board
- Autotuning can be sped up by adding more of these development boards.
- micro USB cable
- A computer capable of running a VM hypervisor (VirtualBox or Parallels).
- Vagrant (you will install this in step 4).
-
Clone this repository (use
git clone --recursive
to clone submodules). -
Clone the TVM repo:
git clone --recursive https://github.com/apache/tvm tvm
-
Setup the microTVM Reference VM.
-
NOTE: Use this
vagrant up
command instead of the one given there:$ TVM_PROJECT_DIR=path/to/microtvm-blogpost-eval vagrant up --provider=<provider>
Choose
virtualbox
orparallels
for<provider>
, depending which VM hypervisor is installed.vmware
support isn't available yet, but we're working on it. -
-
Install extra dependencies. SSH to the VM, then, in the
microtvm-blogpost-eval
directory, run:microtvm-blogpost-eval$ pip3 install -r requirements.txt
-
Setup PYTHONPATH (run from
microtvm-blogpost-eval
):$ export PYTHONPATH=$(pwd)/python
-
Attach the USB device to the Reference VM.
You can run the model in a host-driven configuration with the following command:
$ python -m micro_eval.bin.eval cifar10_cnn:micro_dev:data/cifar10-config-validate.json --validate-against=cifar10_cnn:interp:data/cifar10-config-validate.json
This command builds a Zephyr binary and flashes it to the device. It also builds a similar model to run on the host. It drives both models with the same inputs and displays the output in terminal.
This process is captured in the Jupyter notebook in tutorial/standalone_utvm.ipynb
. You can run this as follows:
-
(If running on the Reference VM) In the
tvm
repo, editapps/microtvm/reference-vm/zephyr/Vagrantfile
and add a line as follows:config.vm.network "forwarded_port", guest: 8090, host: 8090
-
Bounce the VM:
$ vagrant halt $ TVM_PROJECT_DIR=path/to/microtvm-blogpost-eval vagrant up
-
SSH to the VM. Be sure to reset the PYTHONPATH:
$ cd /path/to/microtvm-blogpost-eval $ export PYTHONPATH=$(pwd)/python
-
Install jupyter:
$ pip3 install jupyter
-
Launch the notebook. In this blogpost repo, run
$ python -mjupyter notebook --no-browser --port 8090 --ip=0.0.0.0
Copy the 127.0.0.1
URL from your console to your browser. This should bring up Jupyter notebook--navigate to
tutorial/standalone_utvm.ipynb
and you should be set.
You can also run the Relay CIFAR10 model standalone on-device. First, translate the model into C in the standalone project:
$ python -m micro_eval.bin.standalone cifar10_cnn:micro_dev:data/cifar10-config-validate.json
Now, flash the project onto the device using Zephyr commands:
$ cd standalone
$ west build -b <your_board>
$ west flash
To view the model output, first run miniterm
:
$ python -mserial.tools.miniterm /dev/ttyACM2 115200 # NOTE: replace ttyACM2 with your serial port
--- Miniterm on /dev/ttyACM2 115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
Now, reset the board and you should see:
*** Booting Zephyr OS build zephyr-v2.4.0 ***
uTVM Standalone Demo
TVM complete! Output: (varies depending on the model spec, but should match)
NOTE: This section is WIP with the new on-device runtime.
There are a lot of moving pieces here and it's easy for the system to fail. Here I've tried to document some of the problems you can run into, and how to solve them.
Generally speaking, first try to enable debug logs with --log-level=DEBUG
.
You may see cases where the RPC server times out waiting for the session to be established. This can happen for several reasons:
- A communication problem between the board and VM. Sometimes the serial port can drop bytes. Try unplugging the board and re-attaching it several times, which may clear it up.
- A fault occurs on the device at startup. Try editing the script in question and set the
DEBUG
variable toTrue
. Then, open another terminal and launchpython -mtvm.exec.microtvm_debug_shell
. This window will show the debugger while the script is running. Finally, re-run the edited script in your other terminal. You should see GDB appear in themicrotvm_debug_shell
window and the script will pause at a prompt to let you set up the debugger. To solve session establishment problems, try debugging the UART connection or the main() startup.
If the RPC server times out later on, it's likely there's been a fault in operator execution. Use
the approach from point 2 above, and set a breakpoint at TVMFuncCall
or the function in question.