question: Support embedding?

[Todsaporn Banjerdkit]

Summary

refer to https://github.com/second-state/llama-utils/blob/008da32d648ce4bab40fd029a55a3f72068600cf/api-server/llama-api-server/src/backend/mod.rs#L21

I can see embeddings has been comment out, not sure what need to be done here?

Thanks

Appendix

No response

[alabulei1]

@apepkuss Can you please take a look at this issue?

[Todsaporn Banjerdkit]

FYI: I aim to target bert, distrill-bert, jina-bert, maybe we can get an idea from candle

• bert
• distrill-bert
• jina-bert

[Xin Liu]

@katopz The idea is great! We'll add it to the task list, but we cannot guarantee the schedule, as we do not have enough hands to do it now. Thanks a lot!

[Todsaporn Banjerdkit]

Thanks, I think fastembed-rs is a good start for target Wasm, I will give it a try on my free time.

[hydai]

The embedding mode is supported. Please try the latest version.