WasmEdge / WasmEdge

WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime for cloud native, edge, and decentralized applications. It powers serverless apps, embedded functions, microservices, smart contracts, and IoT devices.

Home Page:https://WasmEdge.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

feat: support embedding mode in WASI-NN ggml plugin

hydai opened this issue · comments

Summary

llama.cpp provides APIs for generating embedding vectors from a given model and prompts.
We would like to enable this feature so users using our WASI-NN ggml plugin can retrieve the embedding vector from the metadata.

Details

Check https://github.com/ggerganov/llama.cpp/blob/master/examples/embedding/embedding.cpp
Provide an option in metadata to enable the embedding mode and return the embedding vector inside the metadata field.

Appendix

No response