Unmixing Convolutional Features for Crisp Edge Detection

Requirements

pytorch >= 1.0

torchvision

opencv-python

tqdm

Training & Testing

Data preparation

Download the BSDS500 and the NYUDv2 provided by HED
Place the images to "./data/.
The structure of the data folder should be

./data
   bsds/test/*.jpg
   bsds/train/aug_data*/*.jpg
   bsds/train/aug_gt*/*.png
   bsds/test.lst
   bsds/train.lst
   ------------------------
   nyud/test/Images/*.png
   nyud/train/Images/*/*.png
   nyud/train/GT/*/*.png
   nyud/test.lst
   nyud/train.lst

For NYUDv2 dataset, the following command can be run for data augmentation

python ./data/aug.py

Pretrained Models

Download the pretrained model and unzip the model to "./pretrained/"
Pretrained model for BSDS500
Pretrained model for NYUDv2

Training

Download the pre-trained vgg16 model, and put it to "./pretrained" folder.

python main.py --mode train

Testing

The default testing data is BSDS500. For the inference results of NYUDv2 dataset, one can change the 8th line in configs/init.py as follows.

class Config(object):
    def __init__(self):
        self.data = "nyud"

python main.py --mode test

The output results will be saved to ./output/$dataset_name/single_scale_test/

Evaluation

The evaluation codes are provided in "./eval", which comes from Structured Edge Detection Toolbox and mayorx
For evaluation, one needs to put the "png2mat.py" to the folder contain test results, and run the following command to change png images to testing format.

python png2mat.py

Quantitative Comparison

BSDS500

Method	ODS	OIS
HED(official/retrained)	0.790 / 0.793	0.808 / 0.811
RCF(official/retrained)	0.798 / 0.799	0.815 / 0.815
BDCN(official/retrained)	0.806 / 0.807	0.826 / 0.822
CATS-HED	0.800	0.816
CATS-RCF	0.805	0.822
CATS-BDCN	0.812	0.828

NYUDv2

Method	ODS	OIS
HED(official/retrained)	0.720 / 0.722	0..734 / 0.737
RCF(official/retrained)	0.743 / 0.745	0.757 / 0.759
BDCN(official/retrained)	0.748 / 0.748	0.763 / 0.762
CATS-HED	0.732	0.746
CATS-RCF	0.752	0.765
CATS-BDCN	0.752	0.765

Visualization Results

There are some visualized results in './examples'.

More results can be downloaded from links below.

The visualization results of BSDS500
The visualization results of NYUDv2

Acknowledgment

We acknowledge the effort from the authors of HED, RCF and BDCN on edge detection. Their researches laid the foundation for this work. We thank meteorshowers as this code is based on the reproduced RCF of pytorch version by meteorshowers.

@article{xie2017hed,
author = {Xie, Saining and Tu, Zhuowen},
journal = {International Journal of Computer Vision},
number = {1},
pages = {3--18},
title = {Holistically-Nested Edge Detection},
volume = {125},
year = {2017}
}

@article{liu2019richer,
author = {Liu, Yun and Cheng, Ming-Ming and Hu, Xiaowei and Bian, Jia-Wang and Zhang, Le and Bai, Xiang and Tang, Jinhui},
journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
number = {8},
pages = {1939--1946},
publisher = {IEEE},
title = {Richer Convolutional Features for Edge Detection},
volume = {41},
year = {2019}
}

@inproceedings{he2019bi-directional,
author = {He, Jianzhong and Zhang, Shiliang and Yang, Ming and Shan, Yanhu and Huang, Tiejun},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
pages = {3828--3837},
title = {Bi-Directional Cascade Network for Perceptual Edge Detection},
year = {2019}
}

isbrycee / CATS