GuangmingZhu / ConvLSTMForGR

Redundancy and Attention in Convolutional LSTM for Gesture Recognition

ConvLSTMForGR

Prerequisites

1. Tensorflow-1.2
   
2. The implementation files of the variants of ConvLSTM are in the local dir "patchs". You need merge them with the corresponding files of TF-1.2.
   
   

Get the pretrained models

The trained models can be obtained from the below link:
Link: https://pan.baidu.com/s/1IoqKr_k006El7WnUavKMeQ Password: y1ua

How to use the code

Prepare the data

1. Convert each video files into images.
2. Replace the path "/ssd/dataset" in the files under "dataset_splits"

Training

1. Use training_*.py to train the networks for different datasets and different modalities.
   

Testing

1. Use testing_*.py to evaluate the trained networks on the valid or test subsets of Jester or IsoGD.
   

Citation

Please cite the following paper if you feel this repository useful.
http://ieeexplore.ieee.org/abstract/document/7880648/ http://openaccess.thecvf.com/content_ICCV_2017_workshops/w44/html/Zhang_Learning_Spatiotemporal_Features_ICCV_2017_paper.html

@article{Zhu2018Attention,
  title={Attention in Convolutional LSTM for Gesture Recognition},
  author={Liang Zhang and Guangming Zhu and Lin Mei and Peiyi Shen and Syed Afaq Shah and Mohammed Bennamoun},
  journal={NIPS},
  year={2018}
}
@article{Zhu2017Learning,
  title={Learning Spatiotemporal Features using 3DCNN and Convolutional LSTM for Gesture Recognition},
  author={Liang Zhang and Guangming Zhu and Peiyi Shen and Juan Song and Syed Afaq Shah and Mohammed Bennamoun},
  journal={ICCV},
  year={2017},
  pages={3120-3128}  
}
@article{Zhu2017MultimodalGR,
  title={Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM},
  author={Guangming Zhu and Liang Zhang and Peiyi Shen and Juan Song},
  journal={IEEE Access},
  year={2017},
  volume={5},
  pages={4517-4524}
}

Contact

For any question, please contact

  [e-mail]