Dense and sparse quantization of open source large lange models, (LLama2, Vicuna), based on Hessian space information. Keeping high accurance and breaking the Memeory Wall.
In the field of LLM quantization, I have noticed a variety of recent critical studies that utilize Hessian information from different angles. I am committed to implementing these methods and comparing them using a uniform standard. This approach will not only differentiate and illustrate their unique properties but also enhance my understanding of these techniques.