sarlinpe / Hand-Joint-Recognition

Recognizing individual 2D joint locations in images

Hand Joint Recognition

Recognizing hand joints in images.

Installation

# create a Python virtualenv
. env.sh
# install Python requirements and setup paths
make install

Requires Python 3.6.1. You will be prompted to provide a path to an experiment directory (training and prediction outputs ) and a data directory (dataset and pre-trained weights). Create them wherever you wish and make sure to provide their absolute paths.

Reproducing our results

Dataset

You should have the two files training.h5 and testing.h5 in your data directory.

Pre-trained weights

The encoder is initialized with weights trained on ImageNet and available here.

cd <DATA_DIR> && mkdir resnet_v2_152 && cd resnet_v2_152
wget http://download.tensorflow.org/models/resnet_v2_152_2017_04_14.tar.gz 
tar -xzf resnet_v2_152_2017_04_14.tar.gz

Training

Our top-scoring model can be trained as follow:

cd jointrecog
python experiment.py train config/resnet_fc.yaml resnet_fc

The training can be interrupted at any time using Ctrl+C and the weights will be saved in <EXPER_DIR>/resnet_fc/. This folder also contains the Tensorboard summaries. If your machine has <90GB RAM you will need to disable caching by setting cache_in_memory to false in config/resnet_fc.yaml. Training for 120k iterations (the default) takes approximately 6 hours on our machine.

Predictions

python export_for_kaggle.py config/resnet_fc.yaml resnet_fc --augment

The csv prediction file will be saved as <EXPER_DIR>/resnet_fc.csv. We also provide the predictions of our top-scoring model as data/resnet-152-block4_fc_no-sym_aug_rot.csv.

Credits

This project was developed by Moritz Zimmermann and Paul-Edouard Sarlin as part of the Machine Perception course at ETH Zurich, spring semester 2018.