Does wasmtime support AI model inference? I want to perform resnet50.pth inference in wasmtime, but there is no relevant tutorial content in the document.
lebron8dong opened this issue · comments
Are there any related examples demo?
Thank you so much.
There is the wasi-nn specification implemented by wasmtime. https://github.com/bytecodealliance/wasmtime/tree/main/crates/wasi-nn/examples has a couple of examples.
It's also possible to compile many models to Wasm and run inference. For example, https://github.com/sonos/tract/ compiles to WebAssembly & WASI and can run any ONNX compatible model.
I have an example of running a BERT model here: https://github.com/rockwotj/ai-qa-wasi/tree/repl