michalwols / SemanticSegmentation_DL

Some implementation of semantic segmantation for DL model

Merge something from https://github.com/nightrome/really-awesome-semantic-segmentation

really-awesome-semantic-segmentation

A list of all papers on Semantic Segmentation and the datasets they use. COCO-Stuff dataset.

Announcement: Starting from September 1 we are organizing the public COCO 2017 Stuff Segmentation Challenge. Please consider participating. Winners will be announced at the Joint COCO and Places Recognition workshop at ICCV 2017.

Dataset importance

Details

For details which paper uses which dataset, please open the Google Drive document.

Survey papers

• A 2017 Guide to Semantic Segmentation with Deep Learning by Qure AI [Blog about different sem. segm. methods]
• A Review on Deep Learning Techniques Applied to Semantic Segmentation [Survey paper with a special focus on datasets and the highest performing methods]
• Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art [Survey paper about all aspects of autonomous vehicles, including sem. segm.] [Webpage with a summary of all relevant publications]

Online demos

• CRF as RNN
• SegNet

SemanticSegmentation_DL

Some implementation of semantic segmantation for DL model

Dataset:

voc2012
MSCOCO
CitySpaces
KITTI
Mapillary
ADE20K
PASCAL Context
COCO-Stuff 10K dataset v1.1
2D-3D-S dataset
Mapillary Vistas

# Resources: # Semantic Segmentation 1. Stacked Deconvolutional Network for Semantic Segmentation-2017 [[Paper]](https://arxiv.org/pdf/1708.04943.pdf)
2. Deeplab v3: Rethinking Atrous Convolution for Semantic Image Segmentation-2017(DeeplabV3) [[Paper]](https://arxiv.org/pdf/1706.05587.pdf)
3. Learning Object Interactions and Descriptions for Semantic Image Segmentation-2017 [[Paper]](http://personal.ie.cuhk.edu.hk/~pluo/pdf/wangLLWcvpr17.pdf)
4. Pixel Deconvolutional Networks-2017 [[Code-Tensorflow]](https://github.com/HongyangGao/PixelDCN) [[Paper]](https://arxiv.org/abs/1705.06820)
5. Dilated Residual Networks-2017 [[Paper]](http://vladlen.info/papers/DRN.pdf)
6. A Review on Deep Learning Techniques Applied to Semantic Segmentation-2017 [[Paper]](https://arxiv.org/abs/1704.06857)
7. BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks [[Paper]](https://arxiv.org/abs/1706.02135)
8. ICNet for Real-Time Semantic Segmentation on High-Resolution Images-2017 [[Project]](https://hszhao.github.io/projects/icnet/) [[Code]](https://github.com/hszhao/ICNet) [[Paper]](https://arxiv.org/abs/1704.08545) [[Video]](https://www.youtube.com/watch?v=qWl9idsCuLQ)
9. Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade-2017 [[Paper]](https://arxiv.org/abs/1704.01344)
10. Loss Max-Pooling for Semantic Image Segmentation-2017 [[Paper]](https://arxiv.org/abs/1704.02966)
11. Annotating Object Instances with a Polygon-RNN-2017 [[Project]](http://www.cs.toronto.edu/polyrnn/) [[Paper]](https://arxiv.org/abs/1704.05548)
12. Feature Forwarding: Exploiting Encoder Representations for Efficient Semantic Segmentation-2017 [[Project]](https://codeac29.github.io/projects/linknet/) [[Code-Torch7]](https://github.com/e-lab/LinkNet)
13. Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation-2017 [[Paper]](https://arxiv.org/abs/1704.03593)
14. Adversarial Examples for Semantic Image Segmentation-2017 [[Paper]](https://arxiv.org/abs/1703.01101)
15. Large Kernel Matters - Improve Semantic Segmentation by Global Convolutional Network-2017 [[Paper]](https://arxiv.org/abs/1703.02719)
16. Label Refinement Network for Coarse-to-Fine Semantic Segmentation-2017 [[Paper]](https://www.arxiv.org/abs/1703.00551)
17. **PixelNet: Representation of the pixels, by the pixels, and for the pixels-2017** [[Project]](http://www.cs.cmu.edu/~aayushb/pixelNet/) [[Code-Caffe]](https://github.com/aayushbansal/PixelNet) [[Paper]](https://arxiv.org/abs/1702.06506)
18. LabelBank: Revisiting Global Perspectives for Semantic Segmentation-2017 [[Paper]](https://arxiv.org/abs/1703.09891)
19. Progressively Diffused Networks for Semantic Image Segmentation-2017 [[Paper]](https://arxiv.org/abs/1702.05839)
20. Understanding Convolution for Semantic Segmentation-2017 [[Model-Mxnet]](https://drive.google.com/drive/folders/0B72xLTlRb0SoREhISlhibFZTRmM) [[Paper]](https://arxiv.org/abs/1702.08502) [[Code]](https://github.com/TuSimple/TuSimple-DUC)
21. Predicting Deeper into the Future of Semantic Segmentation-2017 [[Paper]](https://arxiv.org/abs/1703.07684)
22. **Pyramid Scene Parsing Network-2017** [[Project]](https://hszhao.github.io/projects/pspnet/) [[Code-Caffe]](https://github.com/hszhao/PSPNet) [[Paper]](https://arxiv.org/abs/1612.01105) [[Slides]](http://image-net.org/challenges/talks/2016/SenseCUSceneParsing.pdf)
23. FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation-2016 [[Paper]](https://arxiv.org/abs/1612.02649)
24. FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics-2016 [[Code-PyTorch]](https://github.com/GunhoChoi/FusionNet_Pytorch) [[Paper]](https://arxiv.org/abs/1612.05360)
25. RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation-2016 [[Code-MatConvNet]](https://github.com/guosheng/refinenet) [[Paper]](https://arxiv.org/abs/1611.06612)
26. The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation [[Code-Theano]](https://github.com/SimJeg/FC-DenseNet) [[Code-Keras1]](https://github.com/titu1994/Fully-Connected-DenseNets-Semantic-Segmentation) [[Code-Keras2]](https://github.com/0bserver07/One-Hundred-Layers-Tiramisu) [[Paper]](https://arxiv.org/abs/1611.09326)
27. Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes [[Code-Theano]](https://github.com/TobyPDE/FRRN) [[Paper]](https://arxiv.org/abs/1611.08323)
28. PixelNet: Towards a General Pixel-level Architecture-2016 [[Paper]](http://arxiv.org/abs/1609.06694)
29. Recalling Holistic Information for Semantic Segmentation-2016 [[Paper]](https://arxiv.org/abs/1611.08061)
30. Semantic Segmentation using Adversarial Networks-2016 [[Paper]](https://arxiv.org/abs/1611.08408) [[Code-Chainer]](https://github.com/oyam/Semantic-Segmentation-using-Adversarial-Networks)
31. Region-based semantic segmentation with end-to-end training-2016 [[Paper]](http://arxiv.org/abs/1607.07671)
32. Exploring Context with Deep Structured models for Semantic Segmentation-2016 [[Paper]](https://arxiv.org/abs/1603.03183)
33. Better Image Segmentation by Exploiting Dense Semantic Predictions-2016 [[Paper]](https://arxiv.org/abs/1606.01481)
34. Boundary-aware Instance Segmentation-2016 [[Paper]](https://infoscience.epfl.ch/record/227439/files/HayderHeSalzmannCVPR17.pdf)
35. Improving Fully Convolution Network for Semantic Segmentation-2016 [[Paper]](https://arxiv.org/abs/1611.08986)
36. Deep Structured Features for Semantic Segmentation-2016 [[Paper]](https://arxiv.org/abs/1609.07916)
37. Deeplab v2: DeepLab:Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs-2016** [[Project]](http://liangchiehchen.com/projects/DeepLab.html) [[Code-Caffe]](https://bitbucket.org/deeplab/deeplab-public/) [[Code-Tensorflow]](https://github.com/DrSleep/tensorflow-deeplab-resnet) [[Code-PyTorch]](https://github.com/isht7/pytorch-deeplab-resnet) [[Paper]](https://arxiv.org/abs/1606.00915)
38. DeepLab v1: Semantic Image Segmentation With Deep Convolutional Nets and Fully Connected CRFs-2014** [[Code-Caffe1]](https://bitbucket.org/deeplab/deeplab-public/) [[Code-Caffe2]](https://github.com/TheLegendAli/DeepLab-Context) [[Paper]](http://arxiv.org/abs/1412.7062)
39. Deep Learning Markov Random Field for Semantic Segmentation-2016 [[Project]](http://personal.ie.cuhk.edu.hk/~lz013/projects/DPN.html) [[Paper]](https://arxiv.org/abs/1606.07230)
40. Convolutional Random Walk Networks for Semantic Image Segmentation-2016 [[Paper]](https://arxiv.org/abs/1605.07681)
41. ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation-2016 [[Code-Caffe1]](https://github.com/e-lab/ENet-training)[[Code-Caffe2]](https://github.com/TimoSaemann/ENet) [[Paper]](https://arxiv.org/abs/1606.02147) [[Blog]](https://culurciello.github.io/tech/2016/06/20/training-enet.html)
42. High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks-2016 [[Paper]](https://arxiv.org/abs/1604.04339)
43. ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation-2016 [[Paper]](http://arxiv.org/abs/1604.05144)
44. Object Boundary Guided Semantic Segmentation-2016 [[Code-Caffe]](https://github.com/twtygqyy/obg_fcn) [[Paper]](http://arxiv.org/abs/1603.09742)
45. Segmentation from Natural Language Expressions-2016 [[Project]](http://ronghanghu.com/text_objseg/) [[Code-Tensorflow]](https://github.com/ronghanghu/text_objseg) [[Code-Caffe]](https://github.com/Seth-Park/text_objseg_caffe) [[Paper]](http://arxiv.org/abs/1603.06180)
46. Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation-2016 [[Code-Caffe]](https://github.com/kolesman/SEC) [[Paper]](https://arxiv.org/abs/1603.06098)
47. Global Deconvolutional Networks for Semantic Segmentation-2016 [[Paper]](https://arxiv.org/abs/1602.03930)
48. Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network-2015 [[Project]](http://cvlab.postech.ac.kr/research/transfernet/) [[Code-Caffe]](https://github.com/maga33/TransferNet) [[Paper]](http://arxiv.org/abs/1512.07928)
49. Learning Dense Convolutional Embeddings for Semantic Segmentation-2015 [[Paper]](https://arxiv.org/abs/1511.04377)
50. ParseNet: Looking Wider to See Better-2015 [[Code-Caffe]](https://github.com/weiliu89/caffe/tree/fcn) [[Model-Caffe]](https://github.com/BVLC/caffe/wiki/Model-Zoo#parsenet-looking-wider-to-see-better) [[Paper]](http://arxiv.org/abs/1506.04579)
51. Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation-2015 [[Project]](http://cvlab.postech.ac.kr/research/decouplednet/) [[Code-Caffe]](https://github.com/HyeonwooNoh/DecoupledNet) [[Paper]](http://arxiv.org/abs/1506.04924)
52. **SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation-2015** [[Project]](http://mi.eng.cam.ac.uk/projects/segnet/) [[Code-Caffe]](https://github.com/alexgkendall/caffe-segnet) [[Paper]](http://arxiv.org/abs/1511.00561) [[Tutorial1]](http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html) [[Tutorial2]](https://github.com/alexgkendall/SegNet-Tutorial)
53. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling-2015 [[Code-Caffe]](https://github.com/alexgkendall/caffe-segnet) [[Code-Chainer]](https://github.com/pfnet-research/chainer-segnet) [[Paper]](http://arxiv.org/abs/1505.07293)
54. Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform-2015 [[Paper]](https://arxiv.org/abs/1511.03328)
55. Semantic Segmentation with Boundary Neural Fields-2015 [[Code]](https://github.com/gberta/BNF_globalization) [[Paper]](https://arxiv.org/abs/1511.02674)
56. Semantic Image Segmentation via Deep Parsing Network-2015 [[Project]](http://personal.ie.cuhk.edu.hk/~lz013/projects/DPN.html) [[Paper1]](http://arxiv.org/abs/1509.02634) [[Paper2]](http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Liu_Semantic_Image_Segmentation_ICCV_2015_paper.pdf) [[Slides]](http://personal.ie.cuhk.edu.hk/~pluo/pdf/presentation_dpn.pdf)
57. What’s the Point: Semantic Segmentation with Point Supervision-2015 [[Project]](http://vision.stanford.edu/whats_the_point/) [[Code-Caffe]](https://github.com/abearman/whats-the-point1) [[Model-Caffe]](http://vision.stanford.edu/whats_the_point/models.html) [[Paper]](https://arxiv.org/abs/1506.02106)
58. U-Net: Convolutional Networks for Biomedical Image Segmentation-2015 [[Project]](http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/) [[Code+Data]](http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/u-net-release-2015-10-02.tar.gz) [[Code-Keras]](https://github.com/orobix/retina-unet) [[Code-Tensorflow]](https://github.com/jakeret/tf_unet) [[Paper]](http://arxiv.org/abs/1505.04597) [[Notes]](http://zongwei.leanote.com/post/Pa)
59. Learning Deconvolution Network for Semantic Segmentation(DeconvNet)-2015 [[Project]](http://cvlab.postech.ac.kr/research/deconvnet/) [[Code-Caffe]](https://github.com/HyeonwooNoh/DeconvNet) [[Paper]](http://arxiv.org/abs/1505.04366) [[Slides]](http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil722/slides/w06-deconvnet.pdf)
60. Multi-scale Context Aggregation by Dilated Convolutions-2015 [[Project]](http://vladlen.info/publications/multi-scale-context-aggregation-by-dilated-convolutions/) [[Code-Caffe]](https://github.com/fyu/dilation) [[Code-Keras]](https://github.com/nicolov/segmentation_keras) [[Paper]](http://arxiv.org/abs/1511.07122) [[Notes]](http://www.inference.vc/dilated-convolutions-and-kronecker-factorisation/)
61. ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation-2015 [[Code-Theano]](https://github.com/fvisin/reseg) [[Paper]](https://arxiv.org/abs/1511.07053)
62. BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation-2015 [[Paper]](https://arxiv.org/abs/1503.01640)
63. Feedforward semantic segmentation with zoom-out features-2015 [[Code]](https://bitbucket.org/m_mostajabi/zoom-out-release) [[Paper]](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Mostajabi_Feedforward_Semantic_Segmentation_2015_CVPR_paper.pdf) [[Video]](https://www.youtube.com/watch?v=HvgvX1LXQa8)
64. Conditional Random Fields as Recurrent Neural Networks-2015 [[Project]](http://www.robots.ox.ac.uk/~szheng/CRFasRNN.html) [[Code-Caffe1]](https://github.com/torrvision/crfasrnn) [[Code-Caffe2]](https://github.com/martinkersner/train-CRF-RNN) [[Demo]](http://www.robots.ox.ac.uk/~szheng/crfasrnndemo) [[Paper1]](http://www.robots.ox.ac.uk/~szheng/papers/CRFasRNN.pdf) [[Paper2]](http://arxiv.org/abs/1502.03240)
65. Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation-2015 [[Paper]](https://arxiv.org/abs/1504.01013)
66. **Fully Convolutional Networks for Semantic Segmentation-2015** [[Code-Caffe]](https://github.com/shelhamer/fcn.berkeleyvision.org) [[Model-Caffe]](https://github.com/BVLC/caffe/wiki/Model-Zoo#fcn) [[Code-Tensorflow1]](https://github.com/MarvinTeichmann/tensorflow-fcn) [[Code-Tensorflow2]](https://github.com/shekkizh/FCN.tensorflow) [[Code-Chainer]](https://github.com/wkentaro/fcn) [[Code-PyTorch]](https://github.com/wkentaro/pytorch-fcn) [[Paper1]](http://arxiv.org/abs/1411.4038) [[Paper2]](http://arxiv.org/abs/1605.06211) [[Slides1]](https://docs.google.com/presentation/d/1VeWFMpZ8XN7OC3URZP4WdXvOGYckoFWGVN7hApoXVnc) [[Slides2]](http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-pixels.pdf)
67. Deep Joint Task Learning for Generic Object Extraction-2014 [[Project]](http://vision.sysu.edu.cn/projects/deep-joint-task-learning/) [[Code-Caffe]](https://github.com/xiaolonw/nips14_loc_seg_testonly) [[Dataset]](http://objectextraction.github.io/) [[Paper]](http://ss.sysu.edu.cn/~ll/files/NIPS2014_JointTask.pdf)
68. Highly Efficient Forward and Backward Propagation of Convolutional Neural Networks for Pixelwise Classification-2014 [[Code-Caffe]](https://dl.dropboxusercontent.com/u/6448899/caffe.zip) [[Paper]](https://arxiv.org/abs/1412.4526)
69. **Wider or deeper: Revisiting the resnet model for visual recognition** [[Paper]](https://arxiv.org/abs/1611.10080)
70. Describing the Scene as a Whole: Joint Object Detection, Scene Classification and Semantic Segmentation[[Paper]](https://ttic.uchicago.edu/~yaojian/Paper_Holistic.pdf)
71. Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs[[Paper]](http://www.cv-foundation.org/openaccess/content_cvpr_2013/papers/Mottaghi_Analyzing_Semantic_Segmentation_2013_CVPR_paper.pdf)
72. Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding[[Paper]](https://arxiv.org/abs/1502.06344.pdf)
73. Deep Deconvolutional Networks for Scene Parsing[[Paper]](https://arxiv.org/abs/1411.4101)
74. FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos[[Paper]](https://arxiv.org/pdf/1701.05384.pdf)[[Poject]](http://vision.cs.utexas.edu/projects/fusionseg/)
75. Deep Dual Learning for Semantic Image Segmentation [[Paper]](http://personal.ie.cuhk.edu.hk/~pluo/pdf/luoWLWiccv17.pdf)
76. From image-level to pixel level labeling with convolutional networks [[Paper]]()
77. Scene Segmentation with DAG-Recurrent Neural Networks [[Paper]](http://ieeexplore.ieee.org/abstract/document/7940028/)
78. Learning to Segment Every Thing [[Paper]](https://arxiv.org/pdf/1711.10370.pdf)

Video Semantic Segmentation

1. Feature Space Optimization for Semantic Video Segmentation[Paper][Slides]
   

Road Segmentation

1. MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving [Paper]
   
2. self-driving-car-road-segmentation [Link]
   
3. Efficient Deep Models for Monocular Road Segmentation[Paper]
   
4. Semantic Road Segmentation via Multi-scale Ensembles of Learned Features [Paper]
   
5. Distantly Supervised Road Segmentation [Paper]
   
6. Deep Fully Convolutional Networks with Random Data Augmentation for Enhanced Generalization in Road Detection [Paper]
   
7. Real-time category-based and general obstacle detection for autonomous driving [Paper]
   
8. Road Scene Segmentation from a Single Image [Paper]
   

Transferable Semantic Segmentation

1. Weakly Supervised Object Localization Using Things and Stuff Transfer [Paper]
   
2. Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network [Paper]
   

Real-Time Semantic Segmentation

1. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation [Paper]
   
2. ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation-2016 [Code-Caffe1][Code-Caffe2] [Paper] [Blog]
   
3. Efficient Deep Models for Monocular Road Segmentation[Paper]
   
4. Real-Time Coarse-to-fine Topologically Preserving Segmentation[Paper]
   

Part Semantic Segmentation

1. Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing-2017 [Project] [Code-Caffe] [Paper]
   
2. Deep Learning for Human Part Discovery in Images-2016 [Code-Chainer] [Paper]
   
3. A CNN Cascade for Landmark Guided Semantic Part Segmentation-2016 [Project] [Paper]
   
4. Deep Learning for Semantic Part Segmentation With High-level Guidance-2015 [Paper]
   
5. Neural Activation Constellations-Unsupervised Part Model Discovery with Convolutional Networks-2015 [Paper]
   
6. Human Parsing with Contextualized Convolutional Neural Network-2015 [Paper]
   
7. Part detector discovery in deep convolutional neural networks-2014 [Code] [Paper]
   
8. Hypercolumns for object segmentation and fine-grained localization [Paper]
   

Clothes Parsing

1. Looking at Outfit to Parse Clothing-2017 [Paper]
   
2. Semantic Object Parsing with Local-Global Long Short-Term Memory-2015 [Paper]
   
3. A High Performance CRF Model for Clothes Parsing-2014 [Project] [Code] [Dataset] [Paper]
   
4. Clothing co-parsing by joint image segmentation and labeling-2013 [Project] [Dataset] [Paper]
   
5. Parsing clothing in fashion photographs-2012 [Project] [Paper]
   

Instance Segmentation

1. Pixelwise Instance Segmentation with a Dynamically Instantiated Network-2017 [Paper]
   
2. Semantic Instance Segmentation via Deep Metric Learning-2017 [Paper]
   
3. Mask R-CNN-2017 [Code-Tensorflow] [Paper]
   
4. Pose2Instance: Harnessing Keypoints for Person Instance Segmentation-2017 [Paper]
   
5. Pixelwise Instance Segmentation with a Dynamically Instantiated Network-2017 [Paper]
   
6. Fully Convolutional Instance-aware Semantic Segmentation-2016 [Code] [Paper]
   
7. Instance-aware Semantic Segmentation via Multi-task Network Cascades-2015 [Code] [Paper]
   
8. Recurrent Instance Segmentation-2015 [Project] [Code-Torch7] [Paper] [Poster] [Video]
   
9. Annotating Object Instances with a Polygon-RNN [Paper]

Segment Object Candidates

1. FastMask: Segment Object Multi-scale Candidates in One Shot-2016 [Code-Caffe] [Paper]
   
2. Learning to Refine Object Segments-2016 [Code-Torch] [Paper]
   
3. Learning to Segment Object Candidates-2015 [Code-Torch] [Code-Theano-Keras] [Paper]
   

Foreground Object Segmentation

1. Pixel Objectness-2017 [Project] [Code-Caffe] [Paper]
   
2. A Deep Convolutional Neural Network for Background Subtraction-2017 [Paper]
   

#People: