Multimodal-Aerial-Scene-Recognition
This is the code for paper Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition (ECCV 2020)
Pretrained weight
The pretrain weights of audio and visual net, downloaded from here: https://pan.baidu.com/s/106Y0H5xTXYwi4Zeq2Rw_Og code:5es6 or here: https://zenodo.org/record/4082894
Dataset
We construct a new dataset, named AuDio Visual Aerial sceNe reCognition datasEt (ADVANCE), providing 5075 paired images and sound clips categorized to 13 scenes, for exploring the aerial scene recognition task. You can view the dataset here or directly download the dataset here. Related train/val/test partition can be achieved by the 'data_construction()' function in data/data_partition.py
Usage
The three branchs of cross-modal transfer methods are in the model/ folder, i.g, sq_transfer.py, kl_transfer.py, and bayes_transfer.py. Please extract the sound event knwoledge/prediction using audio_event_extactor.py firstly, then run kl_transfer.py or bayes_transfer.py.
Reference
If you use this repo or the ADVANCE dataset in your research, please cite our paper:
Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition
Di Hu, Xuhong Li, Lichao Mou, Pu Jin, Dong Chen, Liping Jing, Xiaoxiang Zhu, Dejing Dou
ECCV 2020