computer-vision deep-learning generative-model graph-neural-networks machine-learning natural-language-processing semi-supervised-learning

Awesome Semi-Supervised Learning

A curated list of awesome Semi-Supervised Learning resources. Inspired by awesome-deep-vision, awesome-deep-learning-papers, and awesome-self-supervised-learning.

Background

What is Semi-Supervised Learning?

It is a special form of classification. Traditional classifiers use only labeled data (feature / label pairs) to train. Labeled instances however are often difficult, expensive, or time consuming to obtain, as they require the efforts of experienced human annotators. Meanwhile unlabeled data may be relatively easy to collect, but there has been few ways to use them. Semi-supervised learning addresses this problem by using large amount of unlabeled data, together with the labeled data, to build better classifiers. Because semi-supervised learning requires less human effort and gives higher accuracy, it is of great interest both in theory and in practice.

How many semi-supervised learning methods are there?

Many. Some often-used methods include: EM with generative mixture models, self-training, consistency regularization, co-training, transductive support vector machines, and graph-based methods. And with the advent of deep learning, the majority of these methods were adapted and intergrated into existing deep learning frameworks to take advantage of unlabled data.

How do semi-supervised learning methods use unlabeled data?

Semi-supervised learning methods use unlabeled data to either modify or reprioritize hypotheses obtained from labeled data alone. Although not all methods are probabilistic, it is easier to look at methods that represent hypotheses by p(y|x), and unlabeled data by p(x). Generative models have common parameters for the joint distribution p(x,y). It is easy to see that p(x) influences p(y|x). Mixture models with EM is in this category, and to some extent self-training. Many other methods are discriminative, including transductive SVM, Gaussian processes, information regularization, graph-based and the majority of deep learning based methods. Original discriminative training cannot be used for semi-supervised learning, since p(y|x) is estimated ignoring p(x). To solve the problem, p(x) dependent terms are often brought into the objective function, which amounts to assuming p(y|x) and p(x) share parameters

(source: SSL Literature Survey.)

An example of the influence of unlabeled data in semi-supervised learning. (Image source: Wikipedia)

Contributing

If you find any errors, or you wish to add some papers, please feel free to contribute to this list by contacting me or by creating a pull request using the following Markdown format:

- Paper Name. 
  [[pdf]](link) 
  [[code]](link)
  - Author 1, Author 2, and Author 3. *Conference Year*

and adding them to the corresponding markdown file in files/.

Books

Semi-Supervised Learning Book. Olivier Chapelle, Bernhard Schölkopf, Alexander Zien. IEEE Transactions on Neural Networks 2009

Codebase

Surveys & Overview

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, Ian J. Goodfellow. NeurIPS 2018
Semi-Supervised Learning Literature Survey. Xiaojin Zhu. 2008
An Overview of Deep Semi-Supervised Learning. Yassine Ouali, Céline Hudelot, Myriam Tami. 2020
A survey on semi-supervised learning. Jesper E Van Engelen, Holger H Hoos. 2020
A Survey on Deep Semi-Supervised Learning. Xiangli Yang, Zixing Song, Irwin King. 2021

Computer Vision

Image Classification: list of papers here
Semantic and Instance Segmentation: list of papers here
Object Detection: list of papers here
Other tasks: list of papers here

Note that for Image and Object segmentation tasks, we also include weakly-supervised learning methods, that uses weak labels (eg, image classes) for detection and segmentation.