Giters

ngxson / wllama

WebAssembly binding for llama.cpp - Enabling in-browser LLM inference

Home Page:https://huggingface.co/spaces/ngxson/wllama

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support for local webpage use?

twoxfh opened this issue · comments

commented

I do not see a discussions page but was curious if there is any intention to support using wllama without a webserver? Since wasm files can be embedded into html it might make a nice local cross platform desktop app. Thanks for reading and for the project.

commented

In theory it could be possible, but it will be very messy because:

  1. While the compiled wasm can be embedded into html code, it will be embedded as byte array which wastes a lot of space
  2. Due to the way emscripten compiles js file, we will have some headaches patching it to load the correct URL
  3. Multi thread will be impossible due to missing security headers

For these reasons, I have any plans to do this in the future. For now, wllama is mostly for distributing via a CDN platform like netlify. If you plan to make a desktop app, you can rely on ollama, which has multi-platform support while also uses llama.cpp under the hood.

commented

Thanks, I forgot multi-threading would be disabled. My intention was to get around running an executable, and just have kids double click a webpage. For anyone else looking it looks like hosting is still a better option or a desktop Ollama or Llamafile which also allows GPU usage. Will close.