Authors: Tao Zhou, Deng-Ping Fan, Ming-Ming Cheng, Jianbing Shen, Ling Shao.
This is a survey to review related RGB-D SOD models along with benchmark datasets, and provide a comprehensive evaluation for these models. We also collect related review papers for SOD and light field SOD models. If you have papers to recommend or any suggestions, please feel free to contact us.
🏃 🏃 🏃 KEEP UPDATING.
Fig.0: A brief chronology of RGB-D based SOD. The first early RGB-D based SOD work was the DM model, proposed in 2012. Deep learning techniques have been widely applied to RGB-D based SOD since 2017. More details can be found in our paper.
- Related Reviews and Surveys to SOD
- RGB-D SOD Models
- RGB-D SOD Datasets
- Light Field SOD
- Evaluation
- RGB-D SOD Benchmark
- Citation
No. | Year | Pub. | Title | Links |
---|---|---|---|---|
01 | 2015 | IEEE TIP | Salient object detection: A benchmark | Paper/Project |
02 | 2018 | IEEE TCSVT | Review of visual saliency detection with comprehensive information | Paper/Project |
03 | 2018 | ACM TIST | A review of co-saliency detection algorithms: Fundamentals, applications, and challenges | Paper/Project |
04 | 2018 | IEEE TSP | Advanced deep-learning techniques for salient and category-specific object detection: A survey | Paper/Project |
05 | 2018 | IJCV | Attentive systems: A survey | Paper/Project |
06 | 2018 | ECCV | Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground | Paper/Project |
07 | 2019 | CVM | Salient object detection: A survey | Paper/Project |
08 | 2019 | IEEE TNNLS | Object detection with deep learning: A review | Paper/Project |
09 | 2019 | arXiv | Salient Object Detection in the Deep Learning Era: An In-Depth Survey | Paper/Project |
🔥🔥🔥Update (in 2020-09-17)
No. | Year | Model | Pub. | Title | Links |
---|---|---|---|---|---|
🔥 98 | 2020 | DCMF | IEEE TIP | RGBD Salient Object Detection via Disentangled Cross-Modal Fusion | Paper/Project |
97 | 2020 | JL-DCF | arXiv (CVPR extension) | Siamese Network for RGB-D Salient Object Detection and Beyond | Paper/Project |
96 | 2020 | MMNet | ACM MM | MMNet: Multi-Stage and Multi-Scale Fusion Network for RGB-D Salient Object Detection | Paper/Project |
95 | 2020 | DASNet | ACM MM | Is depth really necessary for salient object detection? | Paper/Project |
94 | 2020 | FRDT | ACM MM | Feature Reintegration over Differential Treatment: A Top-down and Adaptive Fusion Network for RGB-D Salient Object Detection | Paper/Project |
93 | 2020 | HANet | Appl. Sci. | Hybrid‐Attention Network for RGB‐D Salient Object Detection | Paper/Project |
92 | 2020 | DQSD | IEEE TIP | Depth Quality Aware Salient Object Detection | Paper/Project |
91 | 2020 | DQAM | arXiv | Knowing Depth Quality In Advance: A Depth Quality Assessment Method For RGB-D Salient Object Detection | Paper/Project |
90 | 2020 | SDSFNet | IEEE TIP | Improved Saliency Detection in RGB-D Images Using Two-phase Depth Estimation and Selective Deep Fusion | Paper/Project |
89 | 2020 | ERLF | IEEE TIP | Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection | Paper/Project |
88 | 2020 | MCINet | arXiv | MCINet: Multi-level Cross-modal Interaction Network for RGB-D Salient Object Detection | Paper/Project |
87 | 2020 | PGAR | ECCV | Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection | Paper/Project |
86 | 2020 | ATSA | ECCV | Asymmetric Two-Stream Architecture for Accurate RGB-D Saliency Detection | Paper/Project |
85 | 2020 | BBS-Net | ECCV | BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network | Paper/Project |
84 | 2020 | CoNet | ECCV | Accurate RGB-D Salient Object Detection via Collaborative Learning | Paper/Project |
83 | 2020 | DANet | ECCV | A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection | Paper/Project |
82 | 2020 | CMMS | ECCV | RGB-D salient object detection with cross-modality modulation and selection | Paper/Project |
81 | 2020 | CAS-GNN | ECCV | Cascade graph neural networks for RGB-D salient object detection | Paper/Project |
80 | 2020 | HDFNet | ECCV | Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection | Paper/Project |
79 | 2020 | CMWNet | ECCV | Cross-modal weighting network for RGB-D salient object detection | Paper/Project |
78 | 2020 | UC-Net | CVPR | UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders | Paper/Project |
77 | 2020 | S2MA | CVPR | Learning selective self-mutual attention for RGB-D saliency detection | Paper/Project |
76 | 2020 | SSF | CVPR | Select, supplement and focus for RGB-D saliency detection | Paper/Project |
75 | 2020 | A2dele | CVPR | A2dele: Adaptive and Attentive Depth Distiller for Efficient RGB-D Salient Object Detection | Paper/Project |
74 | 2020 | JL-DCF | CVPR | JL-DCF: Joint learning and densely-cooperative fusion framework for RGB-D salient object detection | Paper/Project |
73 | 2020 | D3Net | IEEE TNNLS | Rethinking RGB-D salient object detection: models, datasets, and large-scale benchmarks | Paper/Project |
72 | 2020 | RGBS | MTAP | Salient object detection for RGB-D images by generative adversarial network | Paper/Project |
71 | 2020 | GFNet | IEEE SPL | GFNet: Gate fusion network with res2net for detecting salient objects in RGB-D images | Paper/Project |
70 | 2020 | SDF | IEEE TIP | Improved saliency detection in RGB-D images using two-phase depth estimation and selective deep fusion | Paper/Project |
69 | 2020 | ICNet | IEEE TIP | ICNet: Information Conversion Network for RGB-D Based Salient Object Detection | Paper/Project |
68 | 2020 | Triple-Net | IEEE SPL | Triple-complementary network for RGB-D salient object detection | Paper/Project |
67 | 2020 | ASIF-Net | IEEE TCYB | ASIF-Net: Attention steered interweave fusion network for RGB-D salient object detection | Paper/Project |
66 | 2020 | BiANet | IEEE TIP | Bilateral Attention Network for RGB-D Salient Object Detection | Paper/Project |
65 | 2020 | PGHF | IEEE Access | Multi-modal weights sharing and hierarchical feature fusion for rgbd salient object detection | Paper/Project |
64 | 2020 | cmSalGAN | IEEE TMM | cmSalGAN: RGB-D Salient Object Detection with Cross-View Generative Adversarial Networks | Paper/Project |
63 | 2020 | CoCNN | PR | CoCNN: RGB-D deep fusion for stereoscopic salient object detection | Paper/Project |
62 | 2020 | GFNet | Neurocomputing | A cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection | Paper/Project |
61 | 2020 | AttNet | IVC | Attention-guided RGBD saliency detection using appearance information | Paper/Project |
60 | 2020 | SSDP | arXiv | Synergistic saliency and depth prediction for RGB-D saliency detection | Paper/Project |
59 | 2020 | DPANet | arXiv | DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection | Paper/Project |
58 | 2019 | DSD | JVCIR | Depth-aware saliency detection using convolutional neural networks | Paper/Project |
57 | 2019 | DMRA | ICCV | Depth-induced Multi-scale Recurrent Attention Network for Saliency Detection | Paper/Project |
56 | 2019 | CPFP | CVPR | Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection | Paper/Project |
55 | 2019 | EPM | IEEE Access | Co-saliency detection for rgbd images based on effective propagation mechanism | Paper/Project |
54 | 2019 | AFNet | IEEE Access | Adaptive Fusion for RGB-D Salient Object Detection | Paper/Project |
53 | 2019 | LSF | arXiv | CNN-based RGB-D Salient Object Detection: Learn, Select and Fuse | Paper/Project |
52 | 2019 | DGT | IEEE TCYB | Going from RGB to RGBD saliency: A depth-guided transformation model | Paper/Project |
51 | 2019 | DCMF | IEEE TCYB | Discriminative cross-modal transfer learning and densely cross-level feedback fusion for RGB-D salient object detection | Paper/Project |
50 | 2019 | TANet | IEEE TIP | Three-stream attention-aware network for RGB-D salient object detection | Paper/Project |
49 | 2019 | DCA | IEEE TIP | Saliency detection via depth-induced cellular automata on light field | Paper/Project |
48 | 2019 | MMCI | PR | Multi-modal fusion network with multi-scale multi-path and cross-modal interactions | Paper/Project |
47 | 2019 | PDNet | ICME | Prior-model guided depth-enhanced network for salient object detection | Paper/Project |
46 | 2019 | CAFM | IEEE TSMC | Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction | Paper/Project |
45 | 2019 | DIL | MTAP | Salient object segmentation based on depth-aware image layering | Paper/Project |
44 | 2019 | TSRN | ICIP | Two-stream refinement network for RGB-D saliency detection | Paper/Project |
43 | 2019 | MLF | SPL | RGB-D salient object detection by a CNN with multiple layers fusion | Paper/Project |
42 | 2019 | SSRC | Neurocomputing | Salient object detection for RGB-D image by single stream recurrent convolution neural network | Paper/Project |
41 | 2018 | CDB | Neurocomputing | Stereoscopic saliency model using contrast and depth-guided-background prior | Paper/Project |
40 | 2018 | ACCF | IROS | Attention-Aware Cross-Modal Cross-Level Fusion Network for RGB-D Salient Object Detection | Paper/Project |
39 | 2018 | SCDL | ICDSP | Rgbd salient object detection using spatially coherent deep learning framework | Paper/Project |
38 | 2018 | PCF | CVPR | Progressively complementarityaware fusion network for RGB-D salient object detection | Paper/Project |
37 | 2018 | CTMF | IEEE TCYB | CNNs-based RGB-D saliency detection via cross-view transfer and multiview fusion | Paper/Project |
36 | 2018 | ICS | IEEE TIP | Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation | Paper/Project |
35 | 2018 | HSCS | IEEE TMM | HSCS: Hierarchical sparsity based co-saliencydetection for RGBD images | Paper/Project |
34 | 2017 | ISC | SIVP | An integration of bottom-up and top-down salient cueson rgb-d data: saliency from objectness versus non-objectness | Paper/Project |
33 | 2017 | MCLP | IEEE TCYB | An iterative co-saliency framework for RGBD images | Paper/Project |
32 | 2017 | DF | IEEE TIP | RGBD Salient Object Detection via Deep Fusion | Paper/Project |
31 | 2017 | MDSF | IEEE TIP | Depth-Aware Salient Object Detection and Segmentation via Multiscale Discriminative Saliency Fusion and Bootstrap Learning | Paper/Project |
30 | 2017 | MFF | IEEE SPL | RGB-D saliency object detection via minimum barrier distance transformand saliency fusion | Paper/Project |
29 | 2017 | TPF | ICCVW | A Three-Pathway Psychobiological Framework of Salient Object Detection Using Stereoscopic Technology | Paper/Project |
28 | 2017 | CDCP | ICCVW | An innovative salient object detection using center-dark channel prior | Paper/Project |
27 | 2017 | BED | ICCVW | Learning RGB-D Salient Object Detection using background enclosure, depth contrast, and top-down features | Paper/Project |
26 | 2017 | MFLN | ICCVS | RGB-D Saliency Detection by Multi-stream Late Fusion Network | Paper/Project |
25 | 2017 | M3Net | IROS | M3Net: Multi-scale multi-path multi-modal fusion network and example application to RGB-D salient object detection | Paper/Project |
24 | 2017 | HOSO | DICTA | HOSO: Histogram of Surface Orientation for RGB-D Salient Object Detection | Paper/Project |
23 | 2016 | GM | ACCV | Visual Saliency detection for RGB-D images with generative mode | Paper/Project |
22 | 2016 | DSF | ICASSP | Depth-aware saliency detection using discriminative saliency fusion | Paper/Project |
21 | 2016 | DCI | ICASSP | Saliency analysis based on depth contrast increased | Paper/Project |
20 | 2016 | BF | ICPR | RGB-D saliency detection under Bayesian framework | Paper/Project |
19 | 2016 | DCMC | IEEE SPL | Saliency detection for stereoscopic images based on depth confidence analysis and multiple cues fusion | Paper/Project |
18 | 2016 | SE | ICME | Salient object detection for RGB-D image via saliency evolution | Paper/Project |
17 | 2016 | LBE | CVPR | Local Background Enclosure for RGB-D Salient Object Detection | Paper/Project |
16 | 2016 | PRC | IEEE Access | Improving RGBD Saliency Detection Using Progressive Region Classification and Saliency Fusion | Paper/Project |
15 | 2015 | SF | CAC | Selective features for RGB-D saliency | Paper/Project |
14 | 2015 | MGMR | ICIP | RGB-D saliency detection via mutual guided manifold ranking | Paper/Project |
13 | 2015 | SRD | ICRA | Salient Regions Detection for Indoor Robots using RGB-D Data | Paper/Project |
12 | 2015 | DIC | TVC | Depth incorporating with color improves salient object detection | Paper/Project |
11 | 2015 | SFP | ICIMCS | Salient object detection in RGB-D image based on saliency fusion and propagation | Paper/Project |
10 | 2015 | GP | CVPRW | Exploiting global priors for RGB-D saliency detection | Paper/Project |
09 | 2014 | ACSD | ICIP | Depth saliency based on anisotropic center-surround difference | Paper/Project |
08 | 2014 | DESM | ICIMCS | Depth Enhanced Saliency Detection Method | Paper/Project |
07 | 2014 | LHM | ECCV | RGBD Salient Object Detection: A Benchmark and Algorithms | Paper/Project |
06 | 2014 | SRDS | ICDSP | Salient region detection for stereoscopic images | Paper/Project |
05 | 2013 | SOS | Neurocomputing | Depth really Matters: Improving Visual Salient Region Detection with Depth | Paper/Project |
04 | 2013 | RC | BMVC | Depth really Matters: Improving Visual Salient Region Detection with Depth | Paper/Project |
03 | 2013 | LS | BMVC | An In Depth View of Saliency | Paper/Project |
02 | 2012 | RCM | ICCSE | Depth combined saliency detection based on region contrast model | Paper/Project |
01 | 2012 | DM | ECCV | Depth matters: Influence of depth cues on visual saliency | Paper/Project |
No. | Dataset | Year | Pub. | Size | #Obj. | Types | Resolution | Download |
---|---|---|---|---|---|---|---|---|
1 | STERE | 2012 | CVPR | 1000 | ~One | Internet | [251-1200] * [222-900] | link |
2 | GIT | 2013 | BMVC | 80 | Multiple | Home environment | 640 * 480 | link |
3 | DES | 2014 | ICIMCS | 135 | One | Indoor | 640 * 480 | link |
4 | NLPR | 2014 | ECCV | 1000 | Multiple | Indoor/outdoor | 640 * 480, 480 * 640 | link |
5 | LFSD | 2014 | CVPR | 100 | One | Indoor/outdoor | 360 * 360 | link |
6 | NJUD | 2014 | ICIP | 1985 | ~One | Moive/internet/photo | [231-1213] * [274-828] | link |
7 | SSD | 2017 | ICCVW | 80 | Multiple | Movies | 960 *1080 | link |
8 | DUT-RGBD | 2019 | ICCV | 1200 | Multiple | Indoor/outdoor | 400 * 600 | link |
9 | SIP | 2020 | TNNLS | 929 | Multiple | Person in wild | 992 * 774 | link |
No. | Year | Model | Pub. | Title | Links |
---|---|---|---|---|---|
01 | 2014 | LFS | CVPR | Saliency detection on light field | Paper/Project |
02 | 2015 | WSC | CVPR | A weighted sparse coding framework for saliency detection | Paper/Project |
03 | 2015 | DILF | IJCAI | Saliency detection with a deeper investigationof light field | Paper/Project |
04 | 2016 | RL | ICASSP | Relative location for light field saliency detection | Paper/Project |
05 | 2017 | MA | ACM TOMM | Saliency detection on light field: A multi-cue approach | Paper/Project |
06 | 2017 | BIF | NPL | A two-stage bayesian integration framework for salient object detection on light field | Paper/Project |
07 | 2017 | LFS | IEEE TPAMI | Saliency Detection on Light Field | Paper/Project |
08 | 2017 | RLM | ICIVC | Saliency detection with relative location measure in light field image | Paper/Project |
09 | 2017 | SGDC | CVPR | Salience guided depth calibration for perceptually optimized compressive light field 3D display | Paper/Project |
10 | 2018 | DCA | FiO | Depth-induced cellular automata for light field saliency | Paper/Project |
11 | 2019 | DLLF | ICCV | Deep learning for light field saliency detection | Paper/Project |
12 | 2019 | DLSD | IJCAI | Deep light-field-driven saliency detection from a single view | Paper/Project |
13 | 2019 | Molf | NIPS | Memory-oriented decoder for light field salient object detection | Paper/Project |
14 | 2020 | ERNet | AAAI | Exploit and replace: An asymmetrical two-stream architecture for versatile light field saliency detection | Paper/Project |
15 | 2020 | DCA | IEEE TIP | Saliency detection via depth-induced cellular automata onlight field | Paper/Project |
16 | 2020 | RDFD | MTAP | Region-based depth feature descriptor for saliency detection light field | Paper/Project |
17 | 2020 | LFNet | IEEE TIP | LFNet light field fusion network for salient object detection | Paper/Project |
18 | 2020 | LFDCN | IEEE TIP | Light field saliency detection with deep convolutional networks | Paper/Project |
No. | Dataset | Year | Pub. | Size | Description | Download |
---|---|---|---|---|---|---|
1 | LFSD | 2014 | CVPR | 100 | It contains 60 indoor and 40 outdoor scenes, and most scenes consist of only one salient object | link |
2 | HFUT | 2017 | ACM TOMM | 255 | Most scenes contain multipleobjects that appear within different locations and scales under complex background clutter | link |
3 | HFUT | 2017 | ACM TOMM | 255 | Most scenes contain multipleobjects that appear within different locations and scales under complex background clutter | link |
4 | DUTLF-FS | 2019 | ICCV | 1465 | It contains several challenges, including lower contrast between salient objects and cluttered background, multiple disconnected salient objects, and dark or strong light conditions | link |
5 | DUTLF-MV | 2019 | IJCAI | 1580 | Each light field consists of multi-view images and a corresponding ground truth | link |
6 | Lytro Illum | 2020 | IEEE TIP | 640 | It includes several challenging factors, e.g., inconsistent illumi?nation conditions, and small salient objects existing in a similar or cluttered background | link |
Fig.1: A comprehensive evaluation for 24 representative RGB-D based SOD models, including LHM, ACSD, DESM, GP, LBE, DCMC, SE, CDCP, CDB, DF, PCF, CTMF, CPFP, TANet, AFNet, MMCI, DMRA, D3Net, SSF, A2dele, S2MA, ICNet, JL-DCF, and UC-Net. We obtain the terms of $S_{\alpha}$ and MAE values for the 24 models on five datasets (i.e., STERE, NLPR, LFSD, DES, and SIP ). We report the mean values of $S_{\alpha}$ and MAE across the five datasets. Note that these better models are shown in the upper left corner (\ie, with a larger $S_{\alpha}$ and smaller MAE).
======================= run evaluation code ===============================
- We have computed values of different evaluation metrics for each image of each models and save as '***.mat', and the results can be downloaded from Google Drive or Baidu Drive(extraction code: urra).
- Please unzip the downloaded file 'Sal_Det_Results_24_Models.zip' and put it into the file 'results';
- To run 'run_overall_evaluation.m' (plot Fig.1 )
======================================================================
Fig.2: PR curves for 24 RGB-D based models on STERE, NLPR, LFSD, DES, SIP, GIT, SSD, and NJUD datasets.
Fig.3: F-measures under different thresholds for 24 RGB-D based models on STERE, NLPR, LFSD, DES, SIP, GIT, SSD, and NJUD datasets.
======================= run plot code ===========================
- To run 'run_plot_curves.m' (plot Fig.2 and Fig.3)
=============================================================
The complete RGB-D SOD benchmark can be found in this page: http://dpfan.net/d3netbenchmark/
If you find our survey paper and evaluation code are useful, please cite the following paper:
@article{zhou2020rgbd,
title={RGB-D Salient Object Detection: A Survey},
author={Zhou, Tao and Fan, Deng-Ping and Cheng, Ming-Ming and Shen, Jianbing and Shao, Ling},
journal={arXiv preprint arXiv:2008.00230},
year={2020},
}