Giters

marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Occasional Segmentation Fault

harryjulian opened this issue · comments

commented

I'm using cTransformers with streamlit to provide a UI to an LLM. After running the model once on my machine and closing the application and trying to start up again, I get repeated segmentation faults and occasionally the following error:

GGML_ASSERT: /home/runner/work/ctransformers/ctransformers/models/ggml/llama.cpp:2903: strcmp(res->name, "result_output") == 0

I specified threads=-1 in the argument to the LLM which seemed to briefly fix the issue - but now it's still occuring.

I'm not sure if this is a cTransformers specific issue, related to ggml or even streamlit but thought it was worth posting as I can't seem to fix it at the moment. I'm on an EC2 instance running ubuntu 20.04, using TheBloke's quantized version of Mistral.