bug: Failed to run `openhermes-2.5-mistral-7b` with the `wasi-nn` plugin of `0.13.5`

[Xin Liu]

Summary

The latest wasi-nn plugin failed to run openhermes-2.5-mistral-7b. And the behaviors on macOS-14.2 (m1 pro) and Ubuntu-22.04 (Intel(R) Xeon(R) Platinum 8259CL CPU), respectively, are different:

• On macOS
  

• On Ubuntu
  

Current State

No response

Expected State

No response

Reproduction steps

• Install WasmEdge 0.13.5 + ggml plugin of latest version
• Download openhermes-2.5-mistral-7b

  curl -LO https://huggingface.co/second-state/OpenHermes-2.5-Mistral-7B-GGUF/resolve/main/openhermes-2.5-mistral-7b.Q5_K_M.gguf

• Download llama-chat.wasm

  curl -LO https://github.com/second-state/llama-utils/raw/dev/chat/llama-chat.wasm

• Run the model

  wasmedge --dir .:. --nn-preload default:GGML:AUTO:openhermes-2.5-mistral-7b.Q5_K_M.gguf llama-chat.wasm -p chatml -r '<|im_end|>'

Screenshots

Any logs you want to share for showing the specific issue

No response

Components

Others

WasmEdge Version or Commit you used

0.13.5

Operating system information

Ubuntu 22.04, macOS 14.2

Hardware Architecture

x86_64, aarch64

Compiler flags and options

No response