ZDstandup / video_repres_sts

Pytorch implementation of Self-supervised Video Representation Learning by Uncovering Motion and Appearance Statistics

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Self-supervised Video Reprepresentation Learning by Uncovering Spatio-temporal Statistics

Pytroch implementation of "Self-supervised Video Reprepresentation Learning by Uncovering Spatio-temporal Statistics", an extension of our previous CVPR 2019 paper.

Tensorflow implementation https://github.com/laura-wang/video_repres_mas.

Overview

Framework of the proposed approach.

Given an unlabeled video clip, 14 motion statistical labels and 13 appearance statistical labels are to be regeressed. These labels characterize the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc.

Requirements

  • pytroch >= 1.3.0
  • tensorboardX
  • cv2
  • scipy

Usage

Data preparation

UCF101 dataset

  • Download the original UCF101 dataset from the official website. And then extarct RGB images from videos and finally extract optical flow data using TVL1 method.
  • Or direclty download the pre-processed RGB and optical flow data of UCF101 here provided by feichtenhofer.

Train

python train.py --rgb_prefix RGB_DIR --flow_x_prefix FLOW_X_DIR --flow_y_prefix FLOW_Y_DIR

TODO

Feature evaluation

  • Video Retrieval
  • Dynamic Scene Recognition
  • Action Similarity Labeling

Citation

If you find this repository useful in your research, please consider citing:

@inproceedings{wang2019self,
  title={Self-Supervised Spatio-Temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics},
  author={Wang, Jiangliu and Jiao, Jianbo and Bao, Linchao and He, Shengfeng and Liu, Yunhui and Liu, Wei},
  booktitle={CVPR},
  pages={4006--4015},
  year={2019}
}

About

Pytorch implementation of Self-supervised Video Representation Learning by Uncovering Motion and Appearance Statistics


Languages

Language:Python 100.0%