Vahe1994 / SpQR

Vahe1994/SpQR Issues

Any update on inference code?
Updated 7 months ago
[ptb perplexity is different from paper]
Updated 8 months ago7
LLaMa 30B loading error
Closed 8 months ago1
Which dataset should I use?
Updated 8 months ago1
has the inference code released?
Updated 8 months ago
Can I save the compressed model for direct inference only?
Closed a year ago3
How to test inference speed?
Updated a year ago4
How evaluation being done without storing quantized weights?
Closed a year ago1
Does permutation order have to be included when saving the quantized model?
Closed a year ago2
Doesn't seem to work for Baichuan-7B
Closed a year ago
CUDA out of memory falcon-40b when using 40Gi A100 GPU
Updated a year ago1
Outlier mask is still permuted when returned
Closed a year ago1
Process killed after eval phase
Updated a year ago4
Post Quantization for nllb-models
Updated a year ago1
Reason for permutation and weights after it is inverted
Closed a year ago4
Provide SpQR trained model weights on OpenLLaMA?
Updated a year ago1
SqueezeLLM
Updated a year ago2
Evaluation code for Falcon models
Updated a year ago1
Why no save?
Closed a year ago2
Will the function of model saving be realized in the future?
Updated a year ago2
model downloading
Closed a year ago1