dsx-ai / TopPaper

Classic papers for beginners, and impact scope for authors.

TopPaper

Classic Papers for Beginners and Impact Scope for Authors.

-----------------------------------------------------------------------------------------
| I am very new to this field, what papers should I read so as to take one step forward? |
-----------------------------------------------------------------------------------------

There have been billions of academic papers around the world. However, maybe only 0.0...01% among them are valuable or are worth reading. Since our limited life has never been forever, TopPaper provide a Top Academic Paper Chart for beginners and reseachers to take one step faster.

Welcome to contribute more subject or valuable (at least you think) papers. Please feel free to pull requests or open an issue.

• 0. Traditional Methods
• 1. CNN - Convolutional Neural Network
  • 1.1 Image Classification
    • 1.1.1 Architecture
    • 1.1.2 Dataset - Augmentation - Trick
  • 1.2 Object Detection
  • 1.3 Object Segmentation
  • 1.4 Re_ID Person Re-Identification
  • 1.5 OCR Optical Character Recognition
  • 1.6 Face Recognition
  • 1.7 NAS Neural Architecture Search
  • 1.8 Image Super_Resolution
  • 1.9 Image Denoising
  • 1.10 Model Compression - Decomposition, Pruning, Quantization, KD
• 2. Transformer in Vision
• 3. Transformer and Self-Attention in NLP
• 4. Others
• Acknowledgement

---

0. Traditional Methods

Abbreviation	Paper	Cited by	Journal	Year	1st Author	1st Affiliation
SIFT	Object Recognition from Local Scale-Invariant Features	20 K	ICCV	1999	David G. Lowe	University of British Columbia
HOG	Histograms of Oriented Gradients for Human Detection	35 K	CVPR	2005	Navneet Dalal	inrialpes
SURF	SURF: Speeded Up Robust Features	18 K	ECCV	2006	Herbert Bay	ETH Zurich
......

1. CNN - Convolutional Neural Network

1.1 Image Classification

1.1.1 Architecture

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
LeNet	Backpropagation applied to handwritten zip code recognition	8.3 K	Neural Computation	1989	Yann Lecun	AT&T Bell Laboratories
LeNet	Gradient-based learning applied to document recognition	35 K	Proceedings of the IEEE	1998	Yann Lecun	AT&T Research Laboratories
ImageNet	ImageNet: A large-scale hierarchical image database	26 K	CVPR	2009	Jia Dengn	Princeton University
AlexNet	ImageNet Classification with Deep Convolutional Neural Networks	79 K	NIPS	2012	Alex Krizhevsky	University of Toronto
ZFNet	Visualizing and Understanding Convolutional Networks	11 K	ECCV	2014	Matthew D Zeiler	New York University
VGGNet	Very Deep Convolutional Networks for Large-Scale Image Recognition	55 K	ICLR	2015	Karen Simonyan	Oxford
GoogLeNet	Going Deeper with Convolutions	29 K	CVPR	2015	Christian Szegedy	Google
GoogLeNet_v2_v3	Rethinking the Inception Architecture for Computer Vision	12 K	CVPR	2016	Christian Szegedy	Google
ResNet	Deep Residual Learning for Image Recognition	74 K	CVPR	2016	Kaiming He	MSRA
DenseNet	Densely Connected Convolutional Networks	15 K	CVPR	2017	Gao Huang	Cornell University
ResNeXt	Aggregated Residual Transformations for Deep Neural Networks	3.9 K	CVPR	2017	Saining Xie	UC San Diego
MobileNet	MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications	7.7 K	arXiv	2017	Andrew G. Howard	Google
SENet	Squeeze-and-Excitation Networks	6.3 K	CVPR	2018	Jie Hu	Momenta
MobileNet_v2	MobileNetV2: Inverted Residuals and Linear Bottlenecks	4.4 K	CVPR	2018	Mark Sandler	Google
ShuffleNet	ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices	2.3 K	CVPR	2018	Xiangyu Zhang	Megvii
ShuffleNet V2	ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design	1.3 K	ECCV	2018	Ningning Ma	Megvii
MobileNet_v3	Searching for MobileNetV3	0.6 K	ICCV	2019	Andrew Howard	Google
EfficientNet	EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks	1.9 K	ICML	2019	Mingxing Tan	Google
GhostNet	GhostNet: More Features from Cheap Operations	0.1 K	CVPR	2020	Kai Han	Huawei Noah
AdderNet	AdderNet: Do We Really Need Multiplications in Deep Learning?	33	CVPR	2020	Hanting Chen	Huawei Noah
Res2Net	Res2Net: A New Multi-scale Backbone Architecture	0.2 K	TPAMI	2021	Shang-Hua Gao	Nankai University

1.1.2 Dataset - Augmentation - Trick

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
BN	Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift	26 K	ICML	2015	Sergey Ioffe	Google
Xavier Init	Understanding the difficulty of training deep feedforward neural networks	12 K	AISTATS	2010	Xavier	Universite de Montreal
Kaiming Init	Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification	11 K	ICCV	2015	Kaiming He	MSRA
LN	Layer Normalization	2.9 K	NIPS	2016	Jimmy Lei Ba	University of Toronto
GN	Group Normalization	1.1 K	ECCV	2018	Yuxin Wu	FAIR
-	Bag of Tricks for Image Classification with Convolutional Neural Networks	361	CVPR	2019	Tong He	Amazon
-	Fixing the train-test resolution discrepancy	122	NeurIPS	2019	Hugo Touvron	FAIR
Auto-Augment	AutoAugment: Learning Augmentation Policies from Data	487	CVPR	2019	Ekin D. Cubuk	Google
-	Fixing the train-test resolution discrepancy: FixEfficientNet	53	Arxiv	2020	Hugo Touvron	FAIR

1.2 Object Detection

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
RCNN	Rich feature hierarchies for accurate object detection and semantic segmentation	17 K	CVPR	2014	Ross Girshick	Berkeley
Fast RCNN	Fast R-CNN	14 K	ICCV	2015	Ross Girshick	Microsoft Research
Faster RCNN	Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks	20 K	NIPS	2015	Shaoqing Ren	USTC, MSRA
SSD	SSD: Single Shot MultiBox Detector	13 K	ECCV	2016	Wei Liu	UNC
YOLO	You Only Look Once: Unified, Real-Time Object Detection	15 K	CVPR	2016	Joseph Redmon	University of Washington
Mask RCNN	Mask R-CNN	10 K	ICCV	2017	Kaiming He	FAIR
DSSD	DSSD : Deconvolutional Single Shot Detector	1.0 K	CVPR	2017	Cheng-Yang Fu	UNC
YOLO9000	YOLO9000: Better, Faster, Stronger.	7.7 K	CVPR	2017	Joseph Redmon	University of Washington
FPN	Feature Pyramid Networks for Object Detection	6.7 K	CVPR	2017	Tsung-Yi Lin	FAIR
Focal Loss	Focal Loss for Dense Object Detection	6.7 K	ICCV	2017	Tsung-Yi Lin	FAIR
Deformable Conv	Deformable Convolutional Networks	1.6 K	ICCV	2017	Jifeng Dai	MSRA
YOLO V3	Yolov3: An incremental improvement	6.9 K	CVPR	2018	Joseph Redmon	University of Washington
ATSS	Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection	0.1 K	CVPR	2020	Shifeng Zhang	CASIA
EfficientDet	EfficientDet: Scalable and Efficient Object Detection	0.3 K	CVPR	2020	Mingxing Tan	Google

1.3 Object Segmentation

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
FCN	Fully Convolutional Networks for Semantic Segmentation	22 K	CVPR	2015	Jonathan Long	UC Berkeley
DeepLab	DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs	7.4 K	ICLR	2015	Liang-Chieh Chen	Google
Unet	U-Net: Convolutional Networks for Biomedical Image Segmentation	24 K	MICCAI	2015	Olaf Ronneberger	University of Freiburg
-	Learning to Segment Object Candidates	0.6 K	NIPS	2015	Pedro O. Pinheiro	FAIR
Dilated Conv	Multi-Scale Context Aggregation by Dilated Convolutions	4.5 K	ICLR	2016	Fisher Y	Princeton University
-	Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network	0.7 K	CVPR	2017	Chao Peng	Tsinghua
RefineNet	RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation	1.6 K	CVPR	2017	Guosheng Lin	The University of Adelaide

1.4 Re_ID Person Re-Identification

1.5 OCR Optical Character Recognition

Abbreviation	Paper	Cited by	Journal	Year	1st Author	1st Affiliation
CTC	Connectionist temporal classifaction: labelling unsegmented sequence data with recurrent neural network	2.9 K	ICML	2006	Alex Graves	IDSIA

1.6 Face Recognition

Abbreviation	Paper	Cited by	Journal	Year	1st Author	1st Affiliation
DeepFace	DeepFace: Closing the Gap to Human-Level Performance in Face Verification	5.3 K	CVPR	2014	Yaniv Taigman	FAIR
DeepID v1	Deep Learning Face Representation from Predicting 10,000 Classes	1.8 K	CVPR	2014	Yi Sun	CUHK
DeepID v2	Deep Learning Face Representation by Joint Identification-Verification	1.9 K	NIPS	2014	Yi Sun	CUHK
FaceNet	FaceNet: A Unified Embedding for Face Recognition and Clustering	7.4 K	CVPR	2015	Florian Schrof	Google
Center Loss	A Discriminative Feature Learning Approach for Deep Face Recognition	2.1 K	ECCV	2016	Yandong Wen	CMU
ArcFace	ArcFace: Additive Angular Margin Loss for Deep Face Recognition	1.3 K	CVPR	2017	Jiankang Deng	Imperial College London
SphereFace	SphereFace: Deep Hypersphere Embedding for Face Recognition	1.3 K	CVPR	2017	Weiyang Liu	Georgia Institute of Technology
CosFace	CosFace: Large Margin Cosine Loss for Deep Face Recognition	0.8 K	CVPR	2018	Hao Wang	Tecent
AM-Softmax Loss	Additive Margin Softmax for Face Verification	0.5 K	Signal Processing Letters	2018	Feng Wang	UESTC

1.7 NAS Neural Architecture Search

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
Darts	DARTS: Differentiable Architecture Search	1.3 K	ICLR	2019	Hanxiao Liu	CMU
-	Neural Architecture Search with Reinforcement Learning	2.5 K	ICLR	2017	Barret Zoph	Google
-	Efficient Neural Architecture Search via Parameter Sharing	1.2 K	ICML	2018	Hieu Pham	Google
-	SNAS: Stochastic Neural Architecture Search	0.3 K	ICLR	2019	Sirui Xie	SenseTime
PC-Darts	PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search	159	ICLR	2020	Yuhui Xu	Huawei

1.8 Image Super_Resolution

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
SRCNN	Image Super-Resolution Using Deep Convolutional Networks	4.1 K	ECCV	2014	Chao Dong	CUHK
ESPCN	Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network	2.4 K	CVPR	2016	Wenzhe Shi	Twitter
FSRCNN	Accelerating the Super-Resolution Convolutional Neural Network	1.3 K	ECCV	2016	Chao Dong	CUHK
VDSR	Accurate Image Super-Resolution Using Very Deep Convolutional Networks	3.5 K	CVPR	2016	Jiwon Kim	Seoul National University
DRCN	Deeply-Recursive Convolutional Network for Image Super-Resolution	1.4 K	CVPR	2016	Jiwon Kim	Seoul National University
EDSR	Enhanced Deep Residual Networks for Single Image Super-Resolution	2.0 K	CVPRW	2017	Bee Lim	Seoul National University
DRRN	Image Super-Resolution via Deep Recursive Residual Network	1.0 K	CVPR	2017	Ying Tai	NJUST
SRDenseNet	Image Super-Resolution Using Dense Skip Connections	0.5 K	ICCV	2017	Tong Tong	Imperial Vision
SRGAN	Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network	5.3 K	CVPR	2017	Christian Ledig	Twitter
LapSRN	Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution	1.1 K	CVPR	2017	Wei-Sheng Lai	1University of California
RDN	Residual Dense Network for Image Super-Resolution	1.1 K	CVPR	2018	Yulun Zhang	Northeastern University
DBPN	Deep Back-Projection Networks For Super-Resolution	0.6 K	CVPR	2018	Muhammad Haris	Toyota Technological Institute
RCAN	Image Super-Resolution Using Very Deep Residual Channel Attention Networks	1.0 K	ECCV	2018	Yulun Zhang	Northeastern University

1.9 Image Denoising

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
CBDNet	Toward Convolutional Blind Denoising of Real Photographs	0.2 K	CVPR	2019	Shi Guo	HIT
-	Learning Deep CNN Denoiser Prior for Image Restoration	0.8 K	CVPR	2017	Kai Zhang	HIT
CnDNN	Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising	2.9 K	TIP	2017	Kai Zhang	HIT
FFDNet	FFDNet: Toward a fast and flexible solution for CNN based image denoising	0.6 K	TIP	2018	Kai Zhang	HIT
SRMD	Learning a Single Convolutional Super-Resolution Network for Multiple Degradations	0.3 K	CVPR	2018	Kai Zhang	HIT
RIDNet	Real Image Denoising with Feature Attention]	87	ICCV	2019	Saeed Anwar	CSIRO
CycleISP	CycleISP: Real Image Restoration via Improved Data Synthesis	28	CVPR	2020	Syed Waqas Zamir	UAE
AINDNet	Transfer Learning from Synthetic to Real-Noise Denoising with Adaptive Instance Normalization	14	CVPR	2020	Yoonsik Kim	Seoul National University

1.10 Model Compression - Decomposition, Pruning, Quantization, KD

Abbreviation	Paper	Cited By	Journal	Year	1st Author	1st Affiliation
-	Tensor Decompositions and Applications	8.5 K	SIAM	2009	Kolda	Sandia National Laboratories
CP-Decomp	Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition	650	ICLR	2015	Vadim Lebedev	Skoltech, Moscow
Tucker-Decomp	Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications	690	ICLR	2016	Yong-Deok Kim	Samsung Electronics
KD	Distilling the Knowledge in a Neural Network	5.8 K	NIPS-W	2014	Geoffrey Hinton	Google
DeepCompression	Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding	4.9K	ICLR	2016	Song Han	Stanford
Fixed Point Quant	Fixed point quantization of deep convolutional networks	0.5 K	ICLR-W	2016	Darryl D. Lin	Qualcomm
DoReFa	DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients	1.1 K	CVPR	2016	Shuchang Zhou	Megvii
Fake Quant	Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference	0.8 K	CVPR	2018	Benoit Jacob	Google
PACT	PACT: Parameterized Clipping Activation for Quantized Neural Networks	300	arXiv	2018	Jungwook Choi	IBM
QIL	Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss	234	CVPR	2019	Sangil Jung	Samsung
DFQ, CLE	data-free quantization through weight equalization and bias correction	160	ICCV	2019	Markus Nagel	Qualcomm AI Research
OCS	Improving Neural Network Quantization without Retraining using Outlier Channel Splitting	151	ICML	Ritchie Zhao	Cornell University
Once for all	Once-for-All: Train One Network and Specialize it for Efficient Deployment	0.1 K	ICLR	2020	Han Cai	MIT
Bit Split	Towards Accurate Post-training Network Quantization via Bit-Split and Stitching	28	ICML	2020	Peisong Wang	Chinese Academy of Sciences
LSQ	Learned Step Size Quantization	110	ICLR	2020	Steven K. Esser	IBM
LSQ+	LSQ+: Improving low-bit quantization through learnable offsets and better initialization	10	CVPR	2020	Yash Bhalgat	Qualcomm
AdaRound	Up or Down? Adaptive Rounding for Post-Training Quantization	37	ICML	2020	Markus Nagel	Qualcomm AI Research
EWGS	Network Quantization with Element-wise Gradient Scaling	-	CVPR	2021	Junghyup Lee	Yonsei University
BRECQ	BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction	52	ICLR	2021	Yuhang Li	UESTC
-	A White Paper on Neural Network Quantization	2	arXiv	2021	Markus	Qualcomm AI Research
QDROP	QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization	-	ICLR	2022	Xiuying Wei	Beihang University
-	Overcoming Oscillations in QAT	1	ICML	2022	Markus Nagel	Qualcomm AI Research

---

2. Transformer in Vision

Abbreviation	Paper	Cited by	Journal	Year	1st Author	1st Affiliation
Image Transformer	Image Transformer	337	ICML	2018	Niki Parmar	Google
-	Attention Augmented Convolutional Networks	191	ICCV	2019	Irwan Bello	Google
DETR	End-to-End Object Detection with Transformers	252	ECCV	2020	Nicolas Carion	Facebook AI
Deit	Training data-efficient image transformers & distillation through attention	57	arXiv	2020	Hugo Touvron	FAIR
i-GPT	Generative Pretraining from Pixels	38	ICML	2020	Mark Chen	OpenAI
Deformable DETR	Deformable DETR: Deformable Transformers for End-to-End Object Detection	12	ICLR	2021	Xizhou Zhu	SenseTime
-	Training data-efficient image transformers & distillation through attention	57	Arxiv	2020	Hugo Touvron	FAIR
ViT	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	175	ICLR	2021	Alexey Dosovitskiy	Google
IPT	Pre-Trained Image Processing Transformer	16	CVPR	2021	Hanting Chen	Huawei Noah
-	A Survey on Visual Transformer	12	Arxiv	2021	Kai Han	Huawei Noah
TNT	Transformer in Transformer	8	Arxiv	2021	Kai Han	Huawei Noah
......

---

3. Transformer and Self-Attention in NLP

Abbreviation	Paper	Cited by	Journal	Year	1st Author	1st Affiliation
Transformer	Attention Is All You Need	19 K	NIPS	2017	Ashish Vaswani	Google
-	Self-Attention with Relative Position Representations	0.5 K	NAACL	2018	Peter Shaw	Google
Bert	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	17 K	NAACL	2019	Jacob Devlin	Google

---

4. Others

......

Acknowledgement

Thanks for the materias and help from Aidong Men, Bo Yang, Zhuqing Jiang, Qishuo Lu, Zhengxin Zeng, Jia'nan Han, Pengliang Tang, Yiyun Zhao, Xian Zhang ......