Optical Flow Guided Feature for Action Recognition, in Pytorch fashion;
Paper reference CVPR2018 OFF for Action Recogniton
- Prepare UCF-101 and HMDB51 dataset follow instruction of tsn-pytorch;
- Put generated train/test split txt files into data folder;
- Prepare pretrained UCF-101 weights from tsn-pytorch;
- Follow data split by official website;
- caffemodel converted tsn pretrained weight link, see model_utils.py for detail information on loading procedure.
- Temporal Segment Network (TSN) & DataLoader, follow tsn-pytorch
- Original Optical Flow Guided Feature, follow caffe prototxt
- Sobel Operator, follow wiki
- Temporal Segment Consensus module in basic_ops.py
- Pytorch version of OFF Network are RGB_OFF.py and Flow_OFF.py
- Rewrite dataset.py (from tsn-pytorch above) to dataset_off.py, for frame sampling interval consistency, as mentioned in the paper Section 4.2, last paragraph;
- RGB OFF weight pth download
- Flow OFF weight pth download
- RGB OFF evaluation score npz download
- Flow OFF evaluation score npz download
- score_fusion.ipynb to fuse scores from RGB and Flow; Try different weight for optimal accuracy;
- Current best accuracy is 95.24%, which is 0.26% less than 95.5% of the original caffe implementaion link
- There is definitely room to boost current 95.24% accuracy, considering training-batch for now is much less than 128 (GPU Memory Limitation, so train-batch in train_off.py is 40-50);
- Weights for other dataset splits and HMDB would not be provided. Feel free to train your own version;
- Code has not been polished, yet functionable;