rustformers / llm

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

Home Page:https://docs.rs/llm/latest/llm/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

WizardCoder llama assert failure

jacohend opened this issue · comments

Trying to run a variety of ggml models from TheBloke leads to this error:
GGML_ASSERT: llama-cpp/ggml.c:6270: ggml_nelements(a) == ne0*ne1*ne2

Wondering if anyone else is experiencing this, and what the issue might be?

Probably another issue with the currently used ggml version, a re-sync with the current main branch of llama.cpp is probably needed.

I actually did that and found a failure on the same assert line. The linked comment said rolling the version back worked best.

I'm wondering if this assert is assuming constant layer sizes, so any modification like The Bloke does might be causing the failure?