maekawatoshiki / altius

Small ONNX inference runtime written in Rust

Altius

Small ONNX inference runtime written in Rust.
Feel free to create issues and discussions!

Requirements

• cargo
• rye

Run

# Download models.
(cd models && ./download.sh)
# Download minimum models.
# (cd models && ./download.sh CI)

# Run examples.
# {mnist, mobilenet, deit, vit} are available.
# You can specify the number of threads for computation by editing the code.
cargo run --release --example mnist
cargo run --release --example mobilenet
cargo run --release --example deit
cargo run --release --example vit

# Experimental CPU backend (that generates code in C)
cargo run --release --example mnist_cpu     -- --iters 10 
cargo run --release --example mobilenet_cpu -- --iters 10 --profile
cargo run --release --example deit_cpu      -- --iters 10 --threads 8 --profile

Run from WebAssembly

Currently, mobilenet v3 runs on web browsers.

cd wasm
cargo install wasm-pack
wasm-pack build --target web
yarn
yarn serve

Run from Python

cd ./crates/altius-py
rye sync --features linux # no --features needed on macOS
rye run maturin develop -r
rye run python mobilenet.py