Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019)

News

[2019/08/27] HigherHRNet is now on ArXiv, which is a bottom-up approach for human pose estimation powerd by HRNet. We will also release code and models at Higher-HRNet-Human-Pose-Estimation, stay tuned!
Our new work High-Resolution Representations for Labeling Pixels and Regions is available at HRNet. Our HRNet has been applied to a wide range of vision tasks, such as image classification, objection detection, semantic segmentation and facial landmark.

Introduction

This is an UNOFFICIAL pytorch implementation of Deep High-Resolution Representation Learning for Human Pose Estimation.

Main Results

Results on MPII val

Arch	Head	Shoulder	Elbow	Wrist	Hip	Knee	Ankle	Mean	Mean@0.1
pose_resnet_50	96.4	95.3	89.0	83.2	88.4	84.0	79.6	88.5	34.0
pose_resnet_101	96.9	95.9	89.5	84.4	88.4	84.5	80.7	89.1	34.0
pose_resnet_152	97.0	95.9	90.0	85.0	89.2	85.3	81.3	89.6	35.0
pose_hrnet_w32	97.1	95.9	90.3	86.4	89.1	87.1	83.3	90.3	37.7

Note:

Flip test is used.
Input size is 256x256
pose_resnet_[50,101,152] is our previous work of Simple Baselines for Human Pose Estimation and Tracking

Results on COCO val2017 with detector having human AP of 56.4 on COCO val2017 dataset

Arch	Input size	#Params	GFLOPs	AP	Ap .5	AP .75	AP (M)	AP (L)	AR	AR .5	AR .75	AR (M)	AR (L)
pose_resnet_50	256x192	34.0M	8.9	0.704	0.886	0.783	0.671	0.772	0.763	0.929	0.834	0.721	0.824
pose_resnet_50	384x288	34.0M	20.0	0.722	0.893	0.789	0.681	0.797	0.776	0.932	0.838	0.728	0.846
pose_resnet_101	256x192	53.0M	12.4	0.714	0.893	0.793	0.681	0.781	0.771	0.934	0.840	0.730	0.832
pose_resnet_101	384x288	53.0M	27.9	0.736	0.896	0.803	0.699	0.811	0.791	0.936	0.851	0.745	0.858
pose_resnet_152	256x192	68.6M	15.7	0.720	0.893	0.798	0.687	0.789	0.778	0.934	0.846	0.736	0.839
pose_resnet_152	384x288	68.6M	35.3	0.743	0.896	0.811	0.705	0.816	0.797	0.937	0.858	0.751	0.863
pose_hrnet_w32	256x192	28.5M	7.1	0.744	0.905	0.819	0.708	0.810	0.798	0.942	0.865	0.757	0.858
pose_hrnet_w32	384x288	28.5M	16.0	0.758	0.906	0.825	0.720	0.827	0.809	0.943	0.869	0.767	0.871
pose_hrnet_w48	256x192	63.6M	14.6	0.751	0.906	0.822	0.715	0.818	0.804	0.943	0.867	0.762	0.864
pose_hrnet_w48	384x288	63.6M	32.9	0.763	0.908	0.829	0.723	0.834	0.812	0.942	0.871	0.767	0.876

Note:

Flip test is used.
Person detector has person AP of 56.4 on COCO val2017 dataset.
pose_resnet_[50,101,152] is our previous work of Simple Baselines for Human Pose Estimation and Tracking.
GFLOPs is for convolution and linear layers only.

Results on COCO test-dev2017 with detector having human AP of 60.9 on COCO test-dev2017 dataset

Arch	Input size	#Params	GFLOPs	AP	Ap .5	AP .75	AP (M)	AP (L)	AR	AR .5	AR .75	AR (M)	AR (L)
pose_resnet_152	384x288	68.6M	35.3	0.737	0.919	0.828	0.713	0.800	0.790	0.952	0.856	0.748	0.849
pose_hrnet_w48	384x288	63.6M	32.9	0.755	0.925	0.833	0.719	0.815	0.805	0.957	0.874	0.763	0.863
pose_hrnet_w48*	384x288	63.6M	32.9	0.770	0.927	0.845	0.734	0.831	0.820	0.960	0.886	0.778	0.877

Note:

Flip test is used.
Person detector has person AP of 60.9 on COCO test-dev2017 dataset.
pose_resnet_152 is our previous work of Simple Baselines for Human Pose Estimation and Tracking.
GFLOPs is for convolution and linear layers only.
pose_hrnet_w48* means using additional data from AI challenger for training.

Environment

The original code was developed using python 3.6 on Ubuntu 16.04. NVIDIA GPUs are needed. The code is developed and tested using 4 NVIDIA P100 GPU cards. The single GPU version was tested on Ubuntu 18.04 with python 3.6 and cuda 10.1 (Other platforms or GPU cards are not fully tested.)

Quick start

Installation

Install pytorch >= v1.0.0 following official instruction. Note that if you use pytorch's version < v1.0.0, you should following the instruction at https://github.com/Microsoft/human-pose-estimation.pytorch to disable cudnn's implementations of BatchNorm layer. We encourage you to use higher pytorch's version(>=v1.0.0)
Clone this repo, and we'll call the directory that you cloned as ${POSE_ROOT}.
Install dependencies:
```
pip install -r requirements.txt
```
Make libs:
```
cd ${POSE_ROOT}/lib
make
```

Install COCOAPI:

# COCOAPI=/path/to/clone/cocoapi
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
cd $COCOAPI/PythonAPI
# Install into global site-packages
make install
# Alternatively, if you do not have permissions or prefer
# not to install the COCO API into global site-packages
python3 setup.py install --user

Note that instructions like # COCOAPI=/path/to/install/cocoapi indicate that you should pick a path where you'd like to have the software cloned and then set an environment variable (COCOAPI in this case) accordingly.

Init output(training model output directory) and log(tensorboard log directory) directory:

mkdir output 
mkdir log

Your directory tree should look like this:

${POSE_ROOT}
├── data
├── experiments
├── lib
├── log
├── models
├── output
├── tools 
├── README.md
└── requirements.txt

Download pretrained models from our model zoo(GoogleDrive or OneDrive)

${POSE_ROOT}
 `-- models
     `-- pytorch
         |-- imagenet
         |   |-- hrnet_w32-36af842e.pth
         |   |-- hrnet_w48-8ef0771d.pth
         |   |-- resnet50-19c8e357.pth
         |   |-- resnet101-5d3b4d8f.pth
         |   `-- resnet152-b121ed2d.pth
         |-- pose_coco
         |   |-- pose_hrnet_w32_256x192.pth
         |   |-- pose_hrnet_w32_384x288.pth
         |   |-- pose_hrnet_w48_256x192.pth
         |   |-- pose_hrnet_w48_384x288.pth
         |   |-- pose_resnet_101_256x192.pth
         |   |-- pose_resnet_101_384x288.pth
         |   |-- pose_resnet_152_256x192.pth
         |   |-- pose_resnet_152_384x288.pth
         |   |-- pose_resnet_50_256x192.pth
         |   `-- pose_resnet_50_384x288.pth
         `-- pose_mpii
             |-- pose_hrnet_w32_256x256.pth
             |-- pose_hrnet_w48_256x256.pth
             |-- pose_resnet_101_256x256.pth
             |-- pose_resnet_152_256x256.pth
             `-- pose_resnet_50_256x256.pth

Data preparation

For MPII data, please download from MPII Human Pose Dataset. The original annotation files are in matlab format. We have converted them into json format, you also need to download them from OneDrive or GoogleDrive. Extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- mpii
    `-- |-- annot
        |   |-- gt_valid.mat
        |   |-- test.json
        |   |-- train.json
        |   |-- trainval.json
        |   `-- valid.json
        `-- images
            |-- 000001163.jpg
            |-- 000003072.jpg

For COCO data, please download from COCO download, 2017 Train/Val is needed for COCO keypoints training and validation. We also provide person detection result of COCO val2017 and test-dev2017 to reproduce our multi-person pose estimation results. Please download from OneDrive or GoogleDrive. Download and extract them under {POSE_ROOT}/data, and make them look like this:

${POSE_ROOT}
|-- data
`-- |-- coco
    `-- |-- annotations
        |   |-- person_keypoints_train2017.json
        |   `-- person_keypoints_val2017.json
        |-- person_detection_results
        |   |-- COCO_val2017_detections_AP_H_56_person.json
        |   |-- COCO_test-dev2017_detections_AP_H_609_person.json
        `-- images
            |-- train2017
            |   |-- 000000000009.jpg
            |   |-- 000000000025.jpg
            |   |-- 000000000030.jpg
            |   |-- ... 
            `-- val2017
                |-- 000000000139.jpg
                |-- 000000000285.jpg
                |-- 000000000632.jpg
                |-- ...

Training and Testing

Testing on MPII dataset using model zoo's models(GoogleDrive or OneDrive)

python tools/test.py --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml TEST.MODEL_FILE models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth

Training on MPII dataset

python tools/train.py --cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml

Testing on COCO val2017 dataset using model zoo's models(GoogleDrive or OneDrive)

python tools/test.py \
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \
    TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth \
    TEST.USE_GT_BBOX False

Training on COCO train2017 dataset

python tools/train.py \
    --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml \

Other applications

Many other dense prediction tasks, such as segmentation, face alignment and object detection, etc. have been benefited by HRNet. More information can be found at Deep High-Resolution Representation Learning.

Citation

If you use this code or models in your research, please cite the original papers with:

@inproceedings{sun2019deep,
  title={Deep High-Resolution Representation Learning for Human Pose Estimation},
  author={Sun, Ke and Xiao, Bin and Liu, Dong and Wang, Jingdong},
  booktitle={CVPR},
  year={2019}
}

@inproceedings{xiao2018simple,
    author={Xiao, Bin and Wu, Haiping and Wei, Yichen},
    title={Simple Baselines for Human Pose Estimation and Tracking},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2018}
}

abhi1kumar / hrnet_pose_single_gpu

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019)

News

Introduction

Main Results

Results on MPII val

Note:

Results on COCO val2017 with detector having human AP of 56.4 on COCO val2017 dataset

Note:

Results on COCO test-dev2017 with detector having human AP of 60.9 on COCO test-dev2017 dataset

Note:

Environment

Quick start

Installation

Data preparation

Training and Testing

Testing on MPII dataset using model zoo's models(GoogleDrive or OneDrive)

Training on MPII dataset

Testing on COCO val2017 dataset using model zoo's models(GoogleDrive or OneDrive)

Training on COCO train2017 dataset

Other applications

Citation

About

Languages