WasmEdge / WasmEdge

WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime for cloud native, edge, and decentralized applications. It powers serverless apps, embedded functions, microservices, smart contracts, and IoT devices.

Home Page:https://WasmEdge.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How can I run wasmedge on Windows11 with GGML plugin

zhanghui-china opened this issue · comments

Summary

on ubuntu, I use
wasmedge --dir .:. --nn-preload default:GGML:AUTO:mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf llama-chat.wasm -p mistral-instruct

It shows:
image

But on windows 11, when i run the same command line:
wasmedge --dir .:. --nn-preload default:GGML:AUTO:mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf llama-chat.wasm -p mistral-instruct

It shows:
19ed33c3e679c38409cc053b04ab387

I've installed wasmedge on windows using "winget install wasmedge"
and i downloaded WasmEdge-plugin-wasi_nn-ggml-0.13.5-darwin_x86_64.tar.gz and decompressed to C:\Program Files\WasmEdge\lib

3de636ebc23ed4fd570f4703f8292a5

How can I run wasmedge on Windows in order to chat with LLM?

Appendix

No response

Hi @zhanghui-china

Currently, we don't support the ggml plugin directly on Windows. If you would like to try it, please use the WSL/WSL2 environment instead. Thanks.

ok,3q。