support native controllers
mmoskal opened this issue · comments
In some cases the controllers are trusted and performance is particularly important. In these cases we should be able to deploy a natively compiled controller. We would need a library that exposes AICI host interface and communicates with aicirt just like normal "wasm-runnner" does.
This is particularly important with speculative decoding