An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
Repository from Github https://github.comoobabooga/AutoGPTQRepository from Github https://github.comoobabooga/AutoGPTQ