Wenhao Wu1,2, Yuxin Song2, Zhun Sun2, Jingdong Wang3, Chang Xu1, Wanli Ouyang3,1
This is the official implementation of our ATM (Arithmetic Temporal Module), which explores the potential of four simple arithmetic operations for temporal modeling.
Our best model can achieve 89.4% Top-1 Acc. on Kinetics-400, 65.6% Top-1 Acc. on Something-Something V1, 74.6% Top-1 Acc. on Something-Something V2!
🔥 I also have other recent video recognition projects that may interest you ✨.
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
Wenhao Wu, Zhun Sun, Wanli Ouyang
Accepted by AAAI 2023 & IJCV 2023 | [Text4Vis Code]
Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models
Wenhao Wu, Xiaohan Wang, Haipeng Luo, Jingdong Wang, Yi Yang, Wanli Ouyang
Accepted by CVPR 2023 | [BIKE Code]
-
TODO: All models and training codes will be released in September! July 14, 2023: 🎉Our ATM has been accepted by ICCV-2023.
The key motivation behind ATM is to explore the potential of simple arithmetic operations to capture auxiliary temporal clues that may be embedded in current video features, without relying on the elaborate design. The ATM can be integrated into both vanilla CNN backbone (e.g., ResNet) and Vision Transformer (e.g., ViT) for video action recognition.
If you use our code in your research or wish to refer to the baseline results, please use the following BibTeX entry😁.
@inproceedings{atm,
title={What Can Simple Arithmetic Operations Do for Temporal Modeling?},
author={Wu, Wenhao and Song, Yuxin and Sun, Zhun and Wang, Jingdong and Xu, Chang and Ouyang, Wanli},
booktitle={Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
year={2023}
}This repository is built upon portions of VideoMAE, CLIP, and EVA. Thanks to the contributors of these great codebases.
For any question, please file an issue or contact Wenhao Wu.