donglixp / awesome-MIM

Reading list for research topics in Masked Image Modeling

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

awesome-MIM

Reading list for research topics in Masked Image Modeling(MIM).

We list the most popular methods for MIM, if I missed something, please submit a request. (Note: We show the date of the first version of Arxiv here. But the link of paper may be not the early version.)

Self-supervied Vision Transformers as backbone models.

Date Method Conference Title Code
2020-xx-xx(maybe 2019) iGPT ICML 2020 Generative Pretraining from Pixels iGPT
2020-10-22 ViT ICLR 2021(Oral) An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale ViT
2021-06-10 MST NeurIPS 2021 MST: Masked Self-Supervised Transformer for Visual Representation None
2021-06-14 BeiT ICLR 2022(Oral) BEiT: BERT Pre-Training of Image Transformers BeiT
2021-11-11 MAE Arxiv 2021 Masked Autoencoders Are Scalable Vision Learners MAE
2021-11-15 iBoT ICLR 2022 iBOT: Image BERT Pre-Training with Online Tokenizer iBoT
2021-11-18 SimMIM Arxiv 2021 SimMIM: A Simple Framework for Masked Image Modeling SimMIM
2021-11-24 PeCo Arxiv 2021 PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers None
2021-12-16 MaskFeat Arxiv 2021 Masked Feature Prediction for Self-Supervised Visual Pre-Training None
2021-12-20 SplitMask Arxiv 2021 Are Large-scale Datasets Necessary for Self-Supervised Pre-training? None
2022-01-31 ADIOS Arxiv 2022 Adversarial Masking for Self-Supervised Learning None
2022-02-07 CAE Arxiv 2022 Context Autoencoder for Self-Supervised Representation Learning None
2022-02-07 CIM Arxiv 2022 Corrupted Image Modeling for Self-Supervised Visual Pre-Training None

Others:

3D.

Date Method Conference Title Code
2021-11-29 Point-BERT Arxiv 2021 Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling Point-BERT

Image generation.

Date Method Conference Title Code
2022-02-08 MaskGIT Arxiv 2022 MaskGIT: Masked Generative Image Transformer None

Video.

Date Method Conference Title Code
2021-03-03 BEVT Arxiv 2022 BEVT: BERT Pretraining of Video Transformers BEVT
ezoic increase your site revenue

About

Reading list for research topics in Masked Image Modeling