Why does the model quantization prompt KILLED at the end?
g558800 opened this issue · comments
Why does the model quantization prompt KILLED at the end?
It's a RAM issue, try to increase the RAM size, or add a memory swap
Resolved. thanks
qwopqwop200 / GPTQ-for-LLaMa
4 bits quantization of LLaMA using GPTQ