能用llama.cpp 4位量化出来跑跑嘛

Question

cgisky1980 opened this issue 2 years ago · comments

没GPU T_T

mabaochang · Answer 1 · Thu Mar 23 2023 12:48:48 GMT+0800 (China Standard Time)

我们正在研发量化功能，将大大降低推理的硬件需求，预计本周发布

Xiangang Li · Answer 2 · Sat Mar 25 2023 23:59:43 GMT+0800 (China Standard Time)

量化模型已经发布，见README.md

能用llama.cpp 4位量化 出来跑跑嘛