LFX Mentorship (Jun-Aug, 2024): Support piper as a new backend of the WASI-NN WasmEdge plugin
hydai opened this issue · comments
Summary
Summary
Motivation
WasmEdge supports PyTorch, TensorFlow Lite, llama.cpp, and more NN backend. Dealing with the text-to-voice is a big thing that we want to achieve. To make it possible, we would like to integrate piper, A fast, local neural text-to-speech system in C++ as a new WASI-NN backend.
Details
- Expected Outcome: A new plugin provides a piper WASI-NN backend, a test suite for validating the plugin, documents, and examples for explaining how to use the plugin.
- Recommended Skills: C++, Wasm
- Mentor(s):
Application link
https://mentorship.lfx.linuxfoundation.org/project/61014739-ac16-4188-bdab-c87c0a502470
Appendix
This project looks very interesting. I really want to attand it! I have rich experience in Rust/Wasm programming and deep learning, so I believe that I am good for this task.
Hi @hydai ,
I am intrigued by this project, If you can suggest to me where to start and any qualifying task with it would be very helpful.
If you are interested in this project and would like to apply for it. Please ensure you can build the piper framework and run the sample applications. Since the whole project is to integrate the piper as one of the WASI-NN backend, the most important part is to understand the piper workflow.
I have built Piper and run several applications on it; everything is working fine so far. Now, I want to begin working on the project and explore more about Piper as per the project's requirements. Has any previous work been done on this project, or do we have to start from scratch? Additionally, if we need to start from scratch, could you please provide some similar references?
Has any previous work been done on this project
None for the Piper integration. But there are lots of different backends for the WASI-NN plugin. You can see the appendix section.
or do we have to start from scratch? Additionally, if we need to start from scratch, could you please provide some similar references?
Same as the previous problem, start from scratch with the Piper part. There is an existing WASI-NN implementation for other backends.
@hydai
Hi There! I am interested in the project and I have good experience with C++. But I am unable to understand how to get started with the project as I have don't know anything about WASM and piper. Any resources you could suggest and can you please clarify what do I need to contribute exactly in this project
@hydai Hi There! I am interested in the project and I have good experience with C++. But I am unable to understand how to get started with the project as I have don't know anything about WASM and piper. Any resources you could suggest and can you please clarify what do I need to contribute exactly in this project
Hey, @Raunak2024
I'm also working around this same, WasmEdge LFX project, I suggest that you use this piper repo as it has lot of examples/samples, links to get you started, rest for WASM you can use this.
@hydai
I am running some simple examples of tutorial
and getting the final output at the end but have no idea why are those initial errors occuring
can someone please guide me on this
Also, I am currently trying to understand the WASI-NN backend implementation for PyTorch. Is there any developer documentation or other resources available that can help me understand the implementation?
@hydai I am running some simple examples of tutorial and getting the final output at the end but have no idea why are those initial errors occuring
can someone please guide me on this
If you are still having this, feel free to reach to me on discord. I had something similar when setting up the RAG server-> Discord ID: angadsinghh
Hey, @Raunak2024 I'm also working around WasmEdge LFX projects, I suggest that you use this piper repo as it has lot of examples/samples, links to get you started, rest for WASM you can use this
Okay! But I was referring to the official site of Wasmedge
@hydai I am running some simple examples of tutorial and getting the final output at the end but have no idea why are those initial errors occuring
can someone please guide me on this
Also, I am currently trying to understand the WASI-NN backend implementation for PyTorch. Is there any developer documentation or other resources available that can help me understand the implementation?
Check out the official repo of WebAssembly once