yuhangYH / UVOT400

Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement

• This repository is the official implementation of our work on Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement.

Paper ArXiv Link | Dataset | Thumbnails | Evaluation Server

Turtle.	Diver.

News:

• Apr. 22, 2024: UVOT400 evaluation server goes live see here.
• Nov. 16, 2023: Added two more trackers (ARTrack, MAT)here.
• Oct. 19, 2023: Added two more trackers (SLT-Track and DropTrack).
• Oct. 16, 2023: Added three more trackers (MixFormer, MixFormerV2, and AiATrack).
• Oct. 12, 2023: Added two new trackers (SimTrack and GRM).
• Oct. 05, 2023: Quickly Benchmark SOTA trackers on your custom videos.
• Aug. 31, 2023: ArXiv Link to paper provided
• Aug. 30, 2023: Paper submitted for publication
• Aug. 07, 2023: Repository made public.
• June 30, 2023: Dataset (Train and Test Set link available)

TODO

• Include all pulled trackers folders
• Provide link to download dataset and annotations (here)
• Experiment environment creation (here)
• Pretrained trackers benchmarking results (here)
• Reproducing our results (here)
• Provide link to download pretrained trackers models (here)
• Provide link to paper (here)
• Provide link to attributes file (here).
• Benchmarking trackers on custom videos here
• Provide dataset thumbnails here.
• UVOT400 dataset evaluation server here.
• Attribute-wise evaluation results here
• Fine-tuned trackers benchmarking results
• Enhanced frames trackers benchmarking results

Our Main Paper Contributions

1. A large and diverse high-quality UVOT400 benchmark dataset is presented, consisting of 400 sequences and 275,000 manually annotated bounding-box frames, introducing 17 distinct tracking attributes with diverse underwater creatures as targets.

2. A large-scale benchmarking of 24 recent SOTA trackers is performed on the proposed dataset, adopting established performance metrics.

3. An UWIE-TR algorithm is introduced. It improves the UVOT performance of SOTA open-air trackers on underwater sequences.

4. The selected SOTA trackers are re-trained on the enhanced version of the proposed dataset resulting in significant performance improvement across all compared trackers.

Our Dataset: UVOT400

Details about the data collection, annotations, domain-specific tracking attributes can be found in our paper.

Links to Datasets

1. Our UVOT400 dataset:

   • Train Set: Download link
   • Test Set: Download link
   • Attributes file: Download link.

2. Our Previous UTB180 Dataset:

   • Kaggle Link.
   • Reference: B. Alawode, Y. Guo, M. Ummar, N. Werghi, J. Dias, A. Mian, and S. Javed, "UTB180: A high-quality benchmark for underwater tracking," in ACCV, 2022.

UVOT400 Evaluation Server

To evaluate your tracker on our dataset, please click here.

NOTE: Both train and test split evaluations are available.

Evaluated Trackers

We have utilized several SOTA trackers for the several experiments we have performed. Links to the github repositories of the trackers are as below (click on the tracker name to go to the github page):

1. Discriminative Correlation Filter-based Trackers:
   • ATOM, DiMP, KYS, PrDiMP, ARDiMP
2. Deep Siamese Trackers
   • SiamFC, SiamRPN, SiamMask, SiamCAR, SiamBAN, SiamGAT, SiamAttn, RBO-SiamRPN++, , KeepTrack
3. Transformer-driven Trackers
   • TrSiam, TrDiMP, STMTrack, TrTr, TransT, Stark, ToMP, RTS, CSWinTT, SparseTT, AutoMatch

For our work, we have pulled the trackers from their respective github repositories.

Experiment Environment Setup

1. Create the python environment

conda create -y --name uvot400 python==3.7.16
conda activate uvot400  

2. Install pytorch and torchvision

pip install torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html

3. Install other packages

pip install -r requirements.txt

4. Build region (for Pysot library)

python setup.py build_ext --inplace

Experiments

For our experiments, we have utilized the success, precision, and normalized precision VOT tracker evaluation metrics. For comparison with GOT10k open-air dataset, the average overlap (AO), success rate 0.50, and 0.75 are utilized.

1. Protocol I: Benchmarking Pre-trained trackers on UVOT400

2. Protocol II: Finetuning and re-benchmarking

3. Protocol III: Image enhancement before tracking

Benchmarking SOTA Trackers on Custom Videos

• This repository also allows you to quickly benchmark SOTA trackers on your custom videos see here.

Aknowledgements

• Thanks to the authors of the trackers for providing the implementations.
• Thanks to the Pysot and Pytracking libraries for providing the tracking evaluation codes.
• We aknowledge the use of eval.ai for creating the evaluation server.
• This work acknowledges the support provided by the Khalifa University of Science and Technology under Faculty Start-Up grants FSU-2022-003 Award No. 8474000401.

Citation

If you find our work useful for your research, please consider citing:

@article{Alawode2023,
archivePrefix = {arXiv},
arxivId = {2308.15816},
author = {Alawode, Basit and Dharejo, Fayaz Ali and Ummar, Mehnaz and Guo, Yuhang and Mahmood, Arif and Werghi, Naoufel and Khan, Fahad Shahbaz and Matas, Jiri and Javed, Sajid},
eprint = {2308.15816},
title = {{Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement}},
url = {http://arxiv.org/abs/2308.15816},
volume = {14},
year = {2023}
}

@inproceedings{alawode2022utb180,
  title={UTB180: A High-quality Benchmark for Underwater Tracking},
  author={Alawode, Basit and Guo, Yuhang and Ummar, Mehnaz and Werghi, Naoufel and Dias, Jorge and Mian, Ajmal and Javed, Sajid},
  booktitle={{ACCV}},
  year={2022}
}