MBNet
Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems (ECCV 2020)
- paper download: https://arxiv.org/pdf/2008.03043.pdf
- the introduction PPT: https://github.com/CalayZhou/MBNet/blob/master/MBNet-3128.pdf
- video demo: https://www.bilibili.com/video/BV1Hi4y137aS
Usage
1. Dependencies
This code is tested on [Ubuntu18.04, tensorflow1.14, keras2.1.6, python3.6,cuda10.0,cudnn7.6].
make sure the GPU enviroment is the same as above (cuda10.0,cudnn7.6), otherwise you may have to compile the
nms
andutils
according to https://github.com/endernewton/tf-faster-rcnn. Besides, check the keras version is keras2.1, i find there may be some mistakes if the keras version is higher. To be as simple as possible, I recommend installing the dependencies with Anaconda as follows:
1. conda create -n python36 python=3.6
2. conda activate python36
3. conda install cudatoolkit=10.0
4. conda install cudnn=7.6
5. conda install tensorflow-gpu=1.14
6. conda install keras=2.1
7. conda install opencv
8. python demo.py
2. Prerequisites
- train data (download to ./data/kaist)
Baidu Cloud(extract code:ABCD
) or KAIST website - test data (download to ./data/kaist_test)
Baidu Cloud(extract code:ABCD
) or Google Drive - ResNet50 pretrained model (download to ./data/models/)
Baidu Cloud(extract code:ABCD
) or Google Drive - MBNet model(download to ./data/models/)
Baidu Cloud(extract code:ABCD
) or Google Drive
3. Demo example
3.1 Demo images
- Check the MBNet model is available at ./data/models/resnet_e7_l224.hdf5
- Run the script:
python demo.py
- The detection result is saved at ./data/kaist_demo/.
3.2 Demo video
- Check the MBNet model is available at ./data/models/resnet_e7_l224.hdf5
- Set weight_path , test_file , lwir_test_file in
demo_video.py
- Run the script:
python demo_video.py
- The detection result videos saved at MBNet directory.
4. Evaluate model performance
- check the MBNet model is available at ./data/models/resnet_e7_l224.hdf5 and the test data is available at ./data/kaist_test.
- Run the script:
python test.py
- The test results are saved at ./data/result/.
- open the KAISTdevkit-matlab-wrapper and run the
demo_test.m
.
5. Train your own model
- Check the ResNet50 pretrained model is available at ./data/models/double_resnet.hdf5 and the train data is available at ./data/kaist.
- Run the script:
python train.py
- The trained models are saved at ./output.
- Evaluate model performance as above.
6. Comparison with other Methods
Please download the Matlab implemented comparison code [Baidu Cloud(extract code: ABCD
) or Google Drive] and run the script according to the README.txt.
7. Acknowledgements
Thanks to Liu Wei, this pipeline is largely built on his ALFNet code available at: https://github.com/liuwei16/ALFNet.
Citation
If you find our work useful in your research, please consider citing:
@inproceedings{MBNet-ECCV2020,
author = {Kailai Zhou and Linsen Chen and Xun Cao},
title = {Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems},
booktitle = ECCV,
year = {2020}
}