WesleyHsieh0806 / Amodal-Expander

Official Training and Inference Code of Amodal Expander, Proposed in Tracking Any Object Amodally

Amodal Expander

Official Training and Inference Code of Amodal Expander, Proposed in Tracking Any Object Amodally.

📙 Project Page | Official Github | 📎 Paper Link | ✏️ Citations

---

Amodal Expander serves as a plug-in module that can “amodalize” any ex-isting detector or tracker with limited (amodal) training data.

📌 Leave a ⭐ in our official repository to keep track of the updates.

---

Table of Contents

• Get Started
• Training and Inference
• Evaluation
• Demo
• PasteNOcclude
• Citations

---

🎒 Get Started

See installation instructions.

🏃 Training and Inference

We augment the SOTA modal tracker GTR with Amodal Expander by fine-tuning on TAO-Amodal dataset.

Please prepare datasets and check our MODEL ZOO for training/inference instructions.

📊 Evaluation

After obtaining the prediction JSON lvis_instances_results.json through the above inference pipeline. You can evaluate the tracker results using our evaluation toolkit.

🤖 Demo

You can test our model on a single video through:

python demo.py --config-file configs/GTR_TAO_Amodal_Expander_PasteNOcclude.yaml \
               --video-input demo/input_video.mp4 \
               --output      demo/output.mp4 \
               --opts        MODEL.WEIGHTS /path/to/Amodal_Expander_PnO_45k.pth

Use --input video_folder/*.jpg instead if the video consists of image frames.

🐇 PasteNOcclude

PasteNOcclude serves as a data augmentation technique to automatically generate more occlusion scenarios. Check the Jupyter demo and implementation details (link 1, link 2, link 3).

Acknowledgement

This repository is built upon Global Tracking Transformer and Detectron2.

LICENSE

Check here for further details.

Citations

@article{hsieh2023tracking,
  title={Tracking any object amodally},
  author={Hsieh, Cheng-Yen and Khurana, Tarasha and Dave, Achal and Ramanan, Deva},
  journal={arXiv preprint arXiv:2312.12433},
  year={2023}
}