yas-sim / human-pose-estimation-2d-demo

Deep learning based human pose estimation demo in Python using Intel OpenVINO toolkit

Human Pose Estimation 2D Demo (Python version)

This program demonstrates how to use the human-pose-estimation-0001 model from OpenVINO Open Model Zoo. The program can extract the pose (a set of coordinations which consists of 18 points from a person) from the people in an image.
The DL model generates heatmaps (HMs) and pair affinity fields (PAFs) form the imput image. The HMs and PAFs are kind of images in 57x32 resolution, and the HMs consists of 18 images and the PAFs consists of 18x2 images.
The user program has to perform image analytics as a postprocess to reconstruct the human pose from the PAFs and HMs.
You can write this process in Python but it's too heavy for Python and could be the bottleneck of the program.
Fortunately, Intel provides this postprocess code in a Python module written in C++. The module is designed for 3d-human-pose-estimation demo program but the basics are the same and it can be reused for reconstructing human pose form the PAFs and HMs generated by a 2D human pose model.
This project contains a small script file to copy and build the pose_extractor module and a Python demo program which demonstrates how to use the pose_extractor in a Python code.

このプログラムはOpenVINO Open Model Zooの human-pose-estimation-0001モデルの使い方のデモプログラムです。プログラムは入力画像から人の姿勢(一人あたり18の座標点)を抽出します。
DLモデル自身は入力画像を推論し、heatmaps (HMs)とpair affinity fields (PAFs)を出力します。HMsとPAFsは一種の画像で57x32の解像度を持ち、HMsは18枚、PAFsは18x2枚の画像からなります。ユーザープログラムは後処理としてPAFsとHMsの画像解析を行い、人の姿勢(各点の座標)を再構築するアルゴリズムを記述する必要があります。
この処理をPythonで記述することも可能ですが、重い処理であり、プログラム全体のボトルネックとなります。 幸いなことにインテルがこの後処理を行うC++で記述されたpose_extractor Pythonモジュールを提供していますのでこれを利用することが可能です。このモジュールは3D-human-pose-estimationデモ用の物ですが基本の処理は同じですのでこれを2Dモデルに再利用することが可能です。
このプロジェクトはpose_extractor後処理モジュールをビルドするための小さなスクリプトと、このモジュールを利用する方法を示すためのPythonコードからなります。

[NEW 09-APR-2022] OpenVINO 2022.1 Support (API 2.0 support)

Human Pose Estimation Result

Required DL Models to Run This Demo

The demo expects the following models in the Intermediate Representation (IR) format:

• human-pose-estimation-0001

You can download this model from OpenVINO Open Model Zoo. In the models.lst is the list of appropriate models for this demo that can be obtained via Model downloader. Please see more information about Model downloader here.

How to Run

0. Prerequisites

• OpenVINO 2020.2
  • If you haven't installed it, go to the OpenVINO web page and follow the Get Started guide to do it.
• Microsoft Visual Studio 2019
  • Community version works. MS VS 2017/2015 may also work. You need to modify the build-poseextractor.bat accordingly.
  • This is requied to build the pose_extractor module.

1. Install dependencies

The demo depends on:

• opencv-python
• numpy

To install all the required Python modules you can use:

(Linux) pip3 install -r requirements.txt
(Win10) pip install -r requirements.txt

2. Download DL models from OMZ

Use Model Downloader to download the required models.

(Linux) python3 $INTEL_OPENVINO_DIR/deployment_tools/tools/model_downloader/downloader.py --list models.lst
(Win10) python "%INTEL_OPENVINO_DIR%\deployment_tools\tools\model_downloader\downloader.py" --list models.lst

3. Build pose_extracor C++ module

The build-poseextractor.[sh|bat] will copy the pose_extractor C++ code from the OpenVINO demo program directory and build it. You'll have pose_extractor.pyd (Win10) or pose_extractor.so (Ubuntu) if you could successfully build the module.

(Linux) 
 $ chmod +x build-poseextractor.sh
 $ ./build-poseextractor.sh

(Win10)
 * Open 'Developer Command Prompt for VS 2019' instead of the 'cmd.exe'. Or the build process will fail because the PATH is not properly set for `msbuild` command.
 > build-poseextractor.bat

4. Run the demo app

Attach a USB webCam as input of the demo program, then run the program. If you want to use a movie file as an input, you can modify the source code to do it.

Also, you can change inference devide by modifying ie.load_network(net_hp, 'CPU') in the code. You can use CPU, GPU, MYRIAD, HDDL, HETERO:FPGA,CPU, MULTI:CPU,GPU, and so on. Please refer to the OpenVINO document web site for details.

(Linux) python3 human-pose-estimation-2d.py
(Win10) python human-pose-estimation-2d.py

Demo Output

The application draws the results on the input image.

Tested Environment

• Windows 10 x64 1909 and Ubuntu 18.04 LTS
• Intel(r) Distribution of OpenVINO(tm) toolkit 2021.3
• Python 3.6.5 x64

See Also

• Using Open Model Zoo demos
• Model Optimizer
• Model Downloader