abbasturkoglu / ADRNet

code and results for 'Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T Tracking'

ADRNet

The official code and results for IJCV paper: Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T Tracking

Framework

• Attribute-Driven Residual Branch (ADRB) aims to build a robust and discriminative appearance model when RGB or T modality is not reliable for tracking. Four heterogeneous attributes, including extreme illumination (EI), occlusion (OCC), motion blur (MB), and thermal crossover (TC), are labeled.
• Attribute Ensemble Network (AENet) aggregates the residual features from different attributes in channel and spatial levels, which consists of two subnetworks: Channel Ensemble Network (CENet) and Spatial Ensemble Network (SENet).

Experiments

Comparison with SOTA on GTOT and RGBT234.

Tracker	GTOT(MSR/MPR)	RGBT234(MSR/MPR)
ADRNet	73.9/90.4	57.1/80.9
CAT(ECCV 20')	71.7/88.9	56.1/80.4
MaCNet(Sensors 20')	71.4/88.0	55.4/79.0
MANet(ICCVW 19')	72.4/89.4	53.9/77.7
DAPNet(ACM MM 19')	70.7/88.2	53.7/76.6

Comparison with VOT2019-RGBT competitors.

Tracker	EAO	Acc	R
JMMAC	0.4826	0.6649	0.8211
ADRNet	0.3959	0.6218	0.7567
SiamDW-T	0.3925	0.6158	0.7839
mfDiMP	0.3879	0.6019	0.8036
FSRPN	0.3553	0.6362	0.7069
MANet	0.3463	0.5823	0.7010
MPAT	0.3180	0.5723	0.7242
CISRDCF	0.2923	0.5215	0.6904
GESBTT	0.2896	0.6163	0.6350

Get Started

Set up Anaconda environment

conda create -n ADRNet python=3.7
conda activate ADRNet
cd $Path_to_ADRNet$
bash install.sh

Run Demo sequence

cd $Path_to_ADRNet$
unzip demp.zip
python Run_test.py

Run RGBT234 and GTOT

cd $Path_to_ADRNet$
python Run_RGBT234.py
python Run_GTOT.py

Training

For training ADRB, you should generate attribute-specific data via

cd $Path_to_ADRNet/data_generation$
python generate_EI_GTOT.py
python generate_MB_GTOT.py
python generate_OCC_GTOT.py
python generate_TC_GTOT.py

Then, generate pkl files via,

cd $Path_to_ADRNet/modules$
python prepro_data_GTOT.py
python prepro_data_RGBT234.py

Finally, you can train the model after setting your data path,

cd $Path_to_ADRNet$
python train_ADRNet.py

Model zoo

The model can be found in google drive and baidu disk(code:56cu). After downloading, you should put it in $Path_to_ADRNet/models/$

Citation

If you feel our work is useful, please cite,

@article{Zhang_IJCV21_ADRNet,
author = {Pengyu Zhang and Dong Wang and Huchuan Lu and Xiaoyun Yang},
title = {Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T Tracking},
journal = IJCV,
volume = {129},
pages = {2714–2729},
year = {2021}
}
If you have any questions, feel free to contract with me