ModelTC / llmc

This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit"

https://arxiv.org/abs/2405.06001

ModelTC/llmc Issues

calib config bs=-1:RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 134 but got size 512 for tensor number 1 in the list
Closed a month ago1
How to use Tensorrt-LLM as backend
Closed a month ago4
Can I use this library for model quantization with only Pytorch+CPU？
Closed 4 months ago3