ltp1995 / SINet

code for "Camouflaged Object Detection" published in CVPR 2020 (Oral)

Camouflaged Object Detection (CVPR2020)

Authors: Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, Ling Shao.

0. Preface

• This repository includes detailed introduction, strong baseline (Search & Identification Net, SINet), and one-key evaluation codes for a brand new field named Camouflaged Object Detection (COD).

• For more information about Camouflaged Object Detection, please visit our Project Page and read the Manuscript (PDF) / Chinese Version (PDF).

• If you have any questions about our paper, feel free to contact Deng-Ping Fan or Ge-Peng Ji via E-mail. And if you are using SINet or evaluation toolbox for your research, please cite this paper (BibTeX)

0.1. 🔥 NEWS 🔥

• [2020/06/11] We re-organize the training set, listed in 2.2. Usage section, please download it again.
• [2020/06/11] 💥 Release train code.
• [2020/05/15] training code is coming soon ...
• [2020/05/05] 💥 Release testing code.
• [2020/04/25] Training/Testing code will be updated soon ...

0.2. Table of Contents

• Camouflaged Object Detection (CVPR2020)
  • 0. Preface
    • 0.1. 🔥 NEWS 🔥
    • 0.2. Table of Contents
  • 1. Task Relationship
  • 2. Proposed Baseline
    • 2.1. Overview
    • 2.2. Usage
  • 3. Results
    • 3.1. Qualitative Comparison
    • 3.2. Quantitative Comparison (Overall/Sub-class)
    • 3.3. Results Download
  • 4. Proposed COD10K Datasets
  • 5. Evaluation Toolbox
  • 6. Potential Applications
  • 7. User Study Test
  • 8. Citation
  • 9. LICENSE
  • 10. Acknowledgements
  • 11. TODO LIST
  • 12. FAQ

1. Task Relationship

Figure 1: Task relationship. One of the most popular directions in computer vision is generic object detection. Note that generic objects can be either salient or camouflaged; camouflaged objects can be seen as difficult cases of generic objects. Typical generic object detection tasks include semantic segmentation and panoptic segmentation (see Fig. 2 b).

Figure 2: Given an input image (a), we present the ground-truth for (b) panoptic segmentation (which detects generic objects including stuff and things), (c) salient instance/object detection (which detects objects that grasp human attention), and (d) the proposed camouflaged object detection task, where the goal is to detect objects that have a similar pattern (e.g., edge, texture, or color) to the natural habitat. In this case, the boundaries of the two butterflies are blended with the bananas, making them difficult to identify. This task is far more challenging than the traditional salient object detection or generic object detection.

References of Salient Object Detection (SOD) benchmark works
[1] Video SOD: Shifting More Attention to Video Salient Object Detection. CVPR, 2019. (Project Page)
[2] RGB SOD: Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground. ECCV, 2018. (Project Page)
[3] RGB-D SOD: Rethinking RGB-D Salient Object Detection: Models, Datasets, and Large-Scale Benchmarks. TNNLS, 2020. (Project Page)
[4] Co-SOD: Taking a Deeper Look at the Co-salient Object Detection. CVPR, 2020. (Project Page)

2. Proposed Baseline

2.1. Overview

Figure 3: Overview of our SINet framework, which consists of two main components: the receptive field (RF) and partial decoder component (PDC). The RF is introduced to mimic the structure of RFs in the human visual system. The PDC reproduces the search and identification stages of animal predation. SA = search attention function described in [71]. See x 4 for details.

2.2. Usage

The training and testing experiments are conducted using PyTorch with a single GeForce RTX TITAN GPU of 24 GB Memory.

Note that our model also supports low memory GPU, which means you can lower the batch size (~419 MB per image in apex-mode=O1, and ~305 MB per image in apex-mode=O2)

1. Configuring your environment (Prerequisites):

   Note that SINet is only tested on Ubuntu OS with the following environments. It may work on other operating systems as well but we do not guarantee that it will.

   • Creating a virtual environment in terminal: conda create -n SINet python=3.6.

   • Installing necessary packages: pip install -r requirements.txt.

   • (Optional: only for training) Installing NVIDIA-Apex for accelerate training process with mixed precision. (Instructions) (Under CUDA-10.0 and Cudnn-7.4).

2. Downloading Training and Testing Sets:

   • download NEW training dataset (COD10K-train + CAMO-train) and move it into ./Dataset/TrainDataset/, which can be found in this download link.

   • downloading NEW testing dataset (COD10K-test + CAMO-test + CHAMELEON) and move it into ./Dataset/TestDataset/, which can be found in this download link.

3. Training Configuration:

   • Assigning your customed path, like --save_model, --train_img_dir, and --train_gt_dir in MyTrain.py.

   • Just run it!

4. Testing Configuration:

   • After you download all the pre-trained model and testing data, just run MyTest.py to generate the final prediction map: replace your trained model directory (--model_path) and assign your the save directory of the inferred mask (--test_save)

   • Note that we re-trained our model (marked as $\diamondsuit$ in the following figure) equipped with mixed training strategy of Apex lib (mode=O1) and get better performance in 40 epoch. Here we provide a new pre-trained model (Baidu Drive (TBD)/Google Drive) here. Later, We will try different backbones based SINet to improve performance and provide more comprehensive comparison.

   
   

5. Evaluation your trained model:

   • One-key evaluation is written in MATLAB code (revised from link), please follow this the instructions in main.m and just run it to generate the evaluation results in ./EvaluationTool/EvaluationResults/Result-CamObjDet/.

3. Results

3.1. Qualitative Comparison

Figure 4: Qualitative results of our SINet and two top-performing baselines on COD10K. Refer to our paper for details.

3.2. Quantitative Comparison (Overall/Sub-class)

Table 1: Quantitative results on different datasets. The best scores are highlighted in bold.

Table 2: Quantitative results of Structure-measure (Sα) for each sub-class in our COD10K dataset-(1/2). The best score of each category is highlighted in bold.

Table 3: Quantitative results of Structure-measure (Sα) for each sub-class in our COD10K dataset-(2/2). The best score of each category is highlighted in bold.

3.3. Results Download

1. Results of our SINet can be found in this download link.

2. Performance of competing methods can be found in this download link.

4. Proposed COD10K Datasets

Figure 5: The extraction of individual samples including 20 sub-classes from our COD10K (2/5)–Aquatic animals.

Figure 6: Annotation diversity and meticulousness in the proposed COD10K dataset. Instead of only providing coarse-grained object-level annotations with the three major types of bias (e.g., Watermark embedded, Coarse annotation, and Occlusion) in prior works, we offer six different annotations, which include edge-level (4rd row), object-level (5rd row), instance-level (6rd row), bounding boxes (7rd row), and attributes (8rd row). Refer to the manuscript for more attribute details.

Figure 7: Regularized quality control during our labeling reverification stage. Strictly adheres to the four major criteria of rejection or acceptance to near the ceiling of annotation accuracy.

COD10K datasets: coming soon.

5. Evaluation Toolbox

We provide complete and fair one-key evaluation toolbox for benchmarking within a uniform standard. Please refer to this link for more information: https://github.com/DengPingFan/CODToolbox

6. Potential Applications

1. Medical (Polyp Segmentation and COVID-19 Infection Segmentation Diagnose)

Figure 8: Lung Infection Segmentation.

​

Figure 9: Example of COVID-19 infected regions in CT axial slice, where the red and green regions denote the GGO, and consolidation, respectively. The images are collected from here. (COVID-19 CT segmentation dataset (link: https://medicalsegmentation.com/covid19/, accessed: 2020-04-11).)

2. Agriculture (locust detection to prevent invasion)

Figure 10: Locust disaster detection.

3. Art (e.g., for photorealistic blending, or recreational art)

Figure 11: The answer can be found at here (Camouflaging an Object from Many Viewpoints, CVPR 2014.)

4. Computer Vision (e.g., for search-and-rescue work, or rare species discovery)

Figure 13: Search and Rescue for saving lives.

5. Underwater Image Enhancement

Figure 14: Please refer to "An Underwater Image Enhancement Benchmark Dataset and Beyond, TIP2019" for more details.

7. User Study Test

--> Click here to explore more interest things (YouTube Link) <--

8. Citation

Please cite our paper if you find the work useful:

@inproceedings{fan2020Camouflage,
title={Camouflaged Object Detection},
author={Fan, Deng-Ping and Ji, Ge-Peng and Sun, Guolei and Cheng, Ming-Ming and Shen, Jianbing and Shao, Ling},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2020}
}

9. LICENSE

• The COD10K Dataset is made available for non-commercial purposes only.

• You will not, directly or indirectly, reproduce, use, or convey the COD10K Dataset or any Content, or any work product or data derived therefrom, for commercial purposes.

10. Acknowledgements

We would like to thank authors of CHAMELEON, CPD1K, and CAMO dataset for their work. They provide tremendous efforts in these dataset to boost this field. We also appreciate image annotators and Wenguan Wang, Geng Chen, Hongsong Wang for insightful feedback and discussion.

11. TODO LIST

If you want to improve the usability or any piece of advice, please feel free to contact me directly (E-mail).

• Support NVIDIA APEX training.

• Support different backbones ( VGGNet, ResNet, ResNeXt Res2Net, iResNet, and ResNeSt etc.)

• Support distributed training.

• Support lightweight architecture and real-time inference, like MobileNet, SqueezeNet.

• Support distributed training

• Add more comprehensive competitors.

12. FAQ

1. If the image cannot be loaded in the page (mostly in the domestic network situations).

   Solution Link

---

⬆ back to top