LFX Mentorship (Mar-May, 2024): Integrate MLX as a new WASI-NN backend

[hydai]

Summary

LLM is a hot topic, there are more and more frameworks to make the execution of LLM faster. WasmEdge already integrated the llama.cpp as one of the backend. And we want to bring more. MLX is an array framework on Apple silicon created by Apple machine learning research. With MLX, we believe it can have a huge improvement on macOS.

Details

• Pre-test: To apply for this mentorship, you MUST finish a pre-test. Pre-test link: #3182
• Expected Outcome: A new plugin provides an MLX WASI-NN backend, a test suite for validating the plugin, documents, and examples for explaining how to use the plugin.
• Recommended Skills: C++, Wasm
• Mentor(s):
  • Hung-Ying Tai (@hydai)
  • dm4 (@dm4)

Appendix

1. MLX: https://github.com/ml-explore/mlx
2. WASI-NN: https://github.com/second-state/wasmedge-wasi-nn
3. WASI-NN llama.cpp backend(check ggml.h and ggml.cpp): https://github.com/WasmEdge/WasmEdge/tree/master/plugins/wasi_nn

[Priyanshi Gaur]

Hi @hydai! I am interested in working on this project under LFX mentorship.

[Tommy shu]

Hi @hydai !I would like to work on this issue under LFX mentorship.

[Aryan Gupta]

Hi @hydai, I am interested in working on this issue as well, having been a part of MLX community, it seems great

[minghuaw]

Just wondering if there is any chance that this could be implemented in rust? I am currently working on a rust binding of mlx.

[hydai]

Just wondering if there is any chance that this could be implemented in rust? I am currently working on a rust binding of mlx.

I don't think we would be interested in an unofficial support binding because we would like to sync with the upstream constantly.