Code for the paper "Generalizing Hand Segmentation in Egocentric Videos with Uncertainty-Guided Model Adaptation" (CVPR2020)
This is the github repository containing the code for the paper "Generalizing Hand Segmentation in Egocentric Videos with Uncertainty-Guided Model Adaptation" by Minjie Cai, Feng Lu and Yoichi Sato.
Currently, the code is under update.
Code usage
Requirements
The code is tested to work correctly with:
- Ubuntu 18.04
- GPU environment
- Anaconda Python 3.7
- Pytorch v0.4.0
- NumPy
- OpenCV
- tqdm
It may also work with other versions of Ubuntu, Python 3 and Pytorch, but we haven't checked.
Dataset preparation
We use several different datasets for model adaptation (or domain adaptation), including: EGTEA, GTEA, EDSH2, EDSH-Kitchen, UTG, YHG, and EgoHands. The UTG and Yale_Human_Grasp datasets are prepared by the paper and included in the repository. Other datasets are public and please download by yourself and arrange them in filefolders like UTG.
Pre-trained model
You can download the hand segmentation model (bayes_rf101_egtea_00040.pth.tar) pre-trained on EGTEA dataset from BaiduDrive code: rvch
Please put the model inside the filefolder according to the filepath of the code.
To generate the pre-trained model for yourself, after preparing the source domain data (EGTEA dataset), run
python train.py --dataset name_of_source_dataset --batchSize your_batchsize_for_training
You can also find the source domain data from the above BaiduDrive.
Running the code
To run the complete experiment, after preparing the data, run
python train_bayesian_adapt.py --dataset name_of_target_dataset --batchSize your_batchsize_for_adaptation
This would adapt the pre-trained segmentation model to the target dataset.
Citation
Please cite the following paper if you feel this repository useful.
@inproceedings{cai2020generalizing,
author = {Minjie Cai and
Feng Lu and
Yoichi Sato},
title = {Generalizing Hand Segmentation in Egocentric Videos with Uncertainty-Guided Model Adaptation},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2020},
}
Contact
For any question, please contact
Minjie Cai: caiminjie@hnu.edu.cn