This is the implementation of the paper: Combining skeleton and accelerometer data for human fine-grained activity recognition and abnormal behaviour detection with deep temporal convolutional networks
The model takes in 2 input streams, which are acceleration and skeleton, and gives a classification result of a 3-second-length window.
- data_processing/noise_filtering.py: Lowpass filter for acceleration data.
- data_processing/preprocess_skeleton.py: Select joints and add angle features to skeleton data.
- model_and_dataset/keras_model_tcn.py: TCN for single-modal data.
- model_and_dataset/keras_model_fusion.py: Fusion for multi-modal features.
- model_and_dataset/keras_model_endtoend.py: Complete fusion model.
The model is built and evaluated on CMDFALL, a multi-modal dataset for HAR.
Original paper of this dataset: A multi-modal multi-view dataset for human fall analysis and preliminary investigation on modality
This dataset is available at: https://www.mica.edu.vn/perso/Tran-Thi-Thanh-Hai/CMDFALL.html
- Python
- Numpy
- Scipy
- Tensorflow 1.14
- Keras-rectified-adam