This repo includes two parts:
- Re-implementation of Temporal Convolutional Networks (For the original version, please refer to Colin Lea's repo).
- The MDP formulation and a customized OpenAI Gym Env. Policy training is implemented with OpenAI Baseline.
All packages should be their versions from around 2018, along with Python 3.6.
- Install scipy, numpy, pandas, matplotlib
- Install pytorch, tensorflow, gym
- Clone this snapshot version of OpenAI Baseline and follow its install instructions. The offical library is under rapid development and often brings breaking changes
- Clone this repo
The code is tested on the JIGWSAWS visual suturing and GTEA. Thanks to Colin Lea for providing pre-processed data for these datasets!
Setttings and hyper-parameters are in config.json.
First setup the experiment in experiment.py. Then:
cd <this repo>
python3 experiment.py
On Google Cloud, this takes about 3 hours (2 hours TCN, 1 hour RL).
CSV files: the final results
Folder result: the results in npy format
Folder graph: plots and figures
Folder tcn_featrues: featrues extracted by TCN (states for RL)
Floder tcn_log: Training log for TCN (Visulize with tensorboard)
Folder tcn_model: Trained models for TCN
Folder trpo_model: Trained models for RL
@InProceedings{10.1007/978-3-030-00937-3_29, author="Liu, Daochang and Jiang, Tingting", title="Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification", booktitle="Medical Image Computing and Computer Assisted Intervention -- MICCAI 2018", year="2018", publisher="Springer International Publishing", address="Cham", pages="247--255", isbn="978-3-030-00937-3" }