Weakly Supervised Semantic Segmentation list

This repository contains lists of state-or-art weakly supervised semantic segmentation works. Papers and resources are listed below according to supervision types.

There are some personal views and notes, just ignore if not interested. (keep updating)

some unsupervised segment proposal methods and datasets here.

CVPR 2018 Tutorial : WSL web&ppt, Part1 ,Part2

Typical weak supervised segmentation problems

No	Supervision	Difficulty	Domain	Core issues
1	Bounding box	middle	annotated classes	transfer learning
2	One-shot segment	middle	similar objects	one-shot learning
3	Image/video label	hard	annotated classes	transfer learning
4	Others	n/a	n/a	n/a

1.Bounding box supervision

Instance semantic segmentation

Learning to Segment Every Thing, CVPR 2018

:Learning weight transfer from well-annotated subset, transfer class-specific weights(output layers) from detection and classification branch, based on Mask-RCNN
Pseudo Mask Augmented Object Detection, CVPR 2018

:State-of-art weakly supervised instance segmentation with bounding box annotation. EM optimizes pseudo mask and segmentation parameter like Boxsup. Graphcut on superpixel is employed to refine pseudo mask.

Simple Does It: Weakly Supervised Instance and Semantic Segmentation, CVPR 2017 [web] [ref-code][supp]

:Grabcut+(HED bounday) and MCG , train foreground segmentation network directly with generated mask semantic segmentaion, sensitive to env(quality) of training images. Check my implementation for pseudo mask generation which is similar to Grabcut+ MCG. But it can't match the performance discribed in paper sup. Opencv version Grabcut perform even worse.
Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation, ICCV 2015

:Based on CRF refine, EM seems not work
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation, ICCV 2015

:Iteratively update parameters and region proposal labels, proposals are selected by network output masks
Deepcut: Object segmentation from bounding box annotations using convolutional neural networks, TMI 2017

Arxiv paper

Learning to Segment via Cut-and-Paste, Arxiv 1803
Adversarial Learning for Semi-Supervised Semantic Segmentation, Arxiv1802, [code]

2.One-Shot segmentation supervision

DAVIS Challenge: http://davischallenge.org/

: Davis17/18(Semi-supervised Video segmentation task), Davis16 is video salient object segmentation without the first frame annotations.

Fast and Accurate Online Video Object Segmentation via Tracking Parts, CVPR 2018(Spotlight) [code]

:state-of-art, 82.4%/1.8s 77.9%/0.6s
OSVOS: One-Shot Video Object Segmentation, CVPR 2017 [web][code]

:milestone, fine-tuning parent network with the first frame mask, 79.8%/10s

3.Image/video label supervision

Self-produced Guidance for Weakly-supervised Object Localization, ECCV 2018
Convolutional Simplex Projection Network (CSPN) for Weakly Supervised Semantic Segmentation, BMVC 2018
Weakly Supervised Instance Segmentation using Class Peak Response, CVPR 2018(Spotlight)

:state-of-art practice for instance seg with only class label.

Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features, CVPR 2018

:Superpixel-> RegionNet(RoI classfier)-> Saliency refine, iteratively update with PixelNet(FCN)
Revisiting Dilated Convolution: A Simple Approach for Weakly- and SemiSupervised Semantic Segmentation, CVPR 2018(Spotlight)
Weakly-Supervised Semantic Segmentation Network With Deep Seeded Region Growing, CVPR 2018 [web][code]
Adversarial Complementary Learning for Weakly Supervised Object Localization, CVPR 2018
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018
Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning, CVPR 2018
Weakly Supervised Semantic Segmentation using Web-Crawled Videos, CVPR 2017(Spotlight) [web]
Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach, CVPR 2017
WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation, CVPR 2017 [web][code]
Learning random-walk label propagation for weakly-supervised semantic segmentation, CVPR 2017(Oral)
Combining Bottom-Up, Top-Down, and Smoothness Cues for Weakly Supervised Image Segmentation, CVPR 2017
Weakly Supervised Semantic Segmentation Using Superpixel Pooling Network, AAAI 2017
Learning from Weak and Noisy Labels for Semantic Segmentation, PAMI 2017
Learning to Segment Human by Watching YouTube, PAMI 2017
Seed, Expand, Constrain: Three Principles for Weakly-Supervised Image Segmentation, ECCV 2016 [code]
Backtracking ScSPM Image Classifier for Weakly Supervised Top-down Saliency, CVPR 2016, TIP 2018 Version
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation, ICCV 2015 [code]
From Image-level to Pixel-level Labeling with Convolutional Networks, CVPR 2015

Resource

Yunchao Wei talk in Chinese about WSL with image label

Arxiv paper

3.1 Deep activation

Propagate method	Papers
Global Max Pooling(GMP)	Is object localization for free? - Weakly-supervised learning with convolutional neural networks,CVPR 2015
Global Average Pooling(GAP)	Learning Deep Features for Discriminative Localization CVPR 2016
Log-sum-exponential Pooling(LSE)	ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks,CVPR 2016
Global Weighted Rank Pooling(GWRP)	SEC ECCV 2016
Global rank Max-Min Pooling(GRP)	WILDCAT, CVPR 2017

3.2 Weakly supervised Detection / Localization(TODO)

Weakly Supervised Region Proposal Network and Object Detection, ECCV 2018
TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection, ECCV 2018
Zigzag Learning for Weakly Supervised Object Detection, CVPR 2018
W2F: A Weakly-Supervised to Fully-Supervised Framework for Object Detection, CVPR 2018
Generative Adversarial Learning Towards Fast Weakly Supervised Detection, CVPR 2018
Min-Entropy Latent Model for Weakly Supervised Object Detection, CVPR 2018
Weakly Supervised Cascaded Convolutional Networks, CVPR 2017
Multiple Instance Detection Network with Online Instance Classifier Refinement, CVPR 2017 [code]

4.Other supervision

Points

Deep Extreme Cut: From Extreme Points to Object Segmentation, CVPR 2018 [web][code]
What's the Point: Semantic Segmentation with Point Supervision, ECCV 2016 [web][code]

Scribbles

Normalized Cut Loss for Weakly-supervised CNN Segmentation, CVPR 2018
ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation, CVPR 2016
Learning to segment under various forms of weak supervision, CVPR 2015

5.Close Related or unpublished work

PCL: Proposal Cluster Learning for Weakly Supervised Object Detection, Arxiv1807 [code]
WebSeg: Learning Semantic Segmentation from Web Searches, Arxiv1803
On Regularized Losses for Weakly-supervised CNN Segmentation, Arxiv1803
Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment, CVPR 2018
Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation, CVPR 2018
Weakly Supervised Salient Object Detection Using Image Labels, AAAI 2018
Weakly Supervised Object Localization on grocery shelves using simple FCN and Synthetic Dataset, Arxiv 1803
Learning Semantic Segmentation with Diverse Supervision, WACV 2018

If some related works are missed, please kindly notice me by dxzhang@zju.edu.cn

iFighting / WeakSupervisedSegmentationList