🔎Transformer-Quantization-Paper-List
Transformer has achieved great success in natural language processing (NLP) tasks and computer vision (CV) tasks over the past two years, showing great potential to surpass CNN in an increasing number of scenarios. However, transformer's special structure and large number of parameters limit its application on resource-constrained devices. More and more research involves how to optimize the transformer model so that it can actually be deployed in more scenarios。
🔈This repository aims to collect the latest transformer model optimization papers, especially those on quantization.
🔖Table of Contents
Papers
▶️ [arxv 2022] Q-ViT: Fully Differentiable Quantization for Vision Transformer. Institute of Automation, Chinese Academy of Sciences▶️ [arxiv 2021] PTQ4ViT: Post-Training Quantization Framework for Vision Transformers. Peking University▶️ [arxv 2021] FQ-ViT: Fully Quantized Vision Transformer without Retraining. MEGVII Technology▶️ [arxv 2021] Understanding and Overcoming the Challenges of Efficient Transformer Quantization Qualcomm AI Research▶️ [arxv 2021] Towards Efficient Post-training Quantization of Pre-trained Language Models The Chinese University of Hong Kong▶️ [PMLR 2021] I-BERT: Integer-only BERT Quantization University of California, Berkeley▶️ [NeurIPS2021] Post-Training Quantization for Vision Transformer. Peking University and Noah’s Ark Lab▶️ [arxv 2020] Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation. Samsung Research