Auto convert transformers models to QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs.
Code based on QuaRot
Auto convert transformers models to QuaRot.
Auto convert transformers models to QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs.
Code based on QuaRot
Auto convert transformers models to QuaRot.
Apache License 2.0