Human pose estimation using OpenPose

• Introduction
• Motivation
• Background
• Installation
• Testing
• Troubleshooting
• References
• Additional Functionality

Introduction

This is a guide to install OpenPose and run a quick demo. However, before we get started, it is crucial to understand what is OpenPose.

OpenPose is an open source library that aims to solve a Computer Vision problem to estimate single human or multi human poses from images or videos in real-time. (Note: There are other implementations that have been developed to accomplish a similar task. See this github repo for a quick summary. Notably, OpenPose is the state of the art, along with AlphaPose and WrnchAI).

Motivation

Before proceeding, check out a live demo here: Real Time Human Pose Estimation in the browser with TensorFlow.js, to get an idea of human pose estimation problem. The live demo is accomplished using PoseNet, which is another library but allows you to skip the tedious task of installing and getting everything up and running.

Now that you got the taste of human estimation problem, let’s install OpenPose and try out some things such as detecting your own pose :).

Background

OpenPose was initially written in C++ and Caffe. Since then, few other variants have been implemented. One of the widely used variant called tf-pose-estimation is based on Tensorflow. It is available in this github repository. We will clone this repository and get it running on our machine in the next section.

However, most of the times, it is not trivial to just follow the readme file on github to run the code. One of the reasons is that because the released package may have several other dependencies and thus, it is not always intuitive as to which version works and which one does not.

So, I would like to highlight the versions I used. (Note, this does not necessarily mean that the OpenPose library does not work on versions other than those listed below. In cases where version compatibility was the root cause of issues I ran into while installing OpenPose, I will specifically call out so in the discussion below. )

• OS: Windows 10 (Note: OpenPose can also run on Ubuntu as specified in the original github repository)
• GPU: Nvidia GeForce GTX 1050 (Implementation is supported on a wide variety of GPUs. Check out the original repo for details.
• GPU driver: 456.38
• Anaconda: Anaconda3-2020.07-Windows-x86_64
• Tensorflow version: 1.14.0
• Python version: 3.7(This will be installed in a new conda environment as shown in the Installation section below)
• git version: 2.28.0.windows.1
• OpenCV version: 4.4.0

Installation

Step 1. Clone the github repository

1. On a Windows PC, open the Command Prompt or the Anaconda Prompt. (I used the Anaconda Prompt)
2. Navigate to a directory where you would like to download the OpenPose project from a github repository. (example, $cd Desktop will make the desktop as our current working directory)
3. Clone the github repo. (Note, make sure you have the git installed beforehand)

$ cd Desktop
$ git clone https://www.github.com/ildoonet/tf-pose-estimation

Step 2: Create a new conda environment

After cloning, notice that there is a new folder created called tf-pose-estimation inside the directory in which you cloned the github repo.

Next, we will create a new conda environment inside this new folder. It is best to work in a new conda environment since we will be downloading various different tools and versions and we don’t want it to mess with our existing projects.

1. Navigate to the tf-pose-estimation folder, that got created after cloning the github repo in Step 1
2. Create a new conda environment and give it a name and specify a python version to install for this environment (In my case, I called new conda environment as ‘tfpose’ and installed python 3.7 version in this environment)
3. Activate the newly created conda environment
4. Install the kernel to the environment

See below!

$ cd tf-pose-estimation
$ conda create - n tfpose python=3.7
$ activate tfpose
$ pip install ipykernel
$ python -m ipykernel install --user --name tfpose --display-name "tfpose"

Step 3: Install the dependencies for OpenPose

The dependencies are specified in the requirements.txt file in the github repo. Install these using a pip command as shown below.

$ pip install -r requirements.txt

Step 4: Install Tensorflow-GPU

Next install the tensorflow-gpu. (Note: I believe OpenPose can also run on non-gpu version of the Tensorflow but I have not tried it.) Let’s install the tensorflow-gpu as shown below.

Note: At the time of this writing, the github repo out of the box, does not work with tensorflow versions >1.14. If you must use tensorflow 2.0, I suggest checking out the tutorials here and and here.

One another thing to note is that, we also need CUDA Toolkit and cuDNN installed but conda inherently takes care of that! This is the most amazing part of conda. I have tried installing CUDA toolkit and cuDNN independently before but it just consumes a lot of time and the process can be intimidating.

$ conda install -c conda-forge tensorflow-gpu = 1.14

Step 5: Install OpenCV

There are multiple ways to install OpenCV. Below, I use a wheel file available here. At this link, search for OpenCV and download a wheel file for the version of python you are using. In my case, since I am using python 3.7, the wheel file I chose was: opencv_python‑4.4.0‑cp37‑cp37m‑win_amd64.whl

Once, the wheel file is downloaded, install it using pip command. (Caution! Before doing the pip install below, make sure you are in the directory where the wheel file was downloaded and saved.)

$ pip install opencv_python‑4.4.0‑cp37‑cp37m‑win_amd64.whl

And this is it! We have installed pretty much everything before proceeding to the next step of testing the code. Make sure to navigate back to the tf-pose-estimation directory if you had to navigate to another directory while installing OpenCV.

Now is a good time to check that we have everything installed correctly.

Step 6: Check to see if all the packages are properly installed

$ conda list

This lists all the packages installed in this conda environment. Below is a snapshot from my system.

Comparing it with the list of items mentioned in the requirements.txt file under the tf-pose-estimation folder, I noticed that requests is not installed for some reason.

Testing/Running OpenPose

Step 1: Active the conda environment

If not already in the newly created conda environment (called ‘tfpose’ in this tutorial), activate this environment before proceeding.

In order to do so, just navigate to the tf-pose-estimation directory and type the following.

$ cd Desktop\tf-pose-estimation
$ activate tfpose

Once activated, you will see (tfpose) as shown below. This indicates that you are now working in a conda environment called tfpose.

Step 2: Run the code

There are several ways to test the code.

Testing Real-time Webcam

First, we will test the pose estimation in real time using the webcam. So, make sure your PC has a working webcam attached. And then, just run the following command.

$ python run_webcam.py --model=mobilenet_thin --resize=432x368 --camera=0

You should now see your webcam feed along with the pose estimation results (stick figures overlaid on the image in the webcam). See example below.

Testing using a single image

Make sure the image to be tested is first placed within the /images folder inside the tf-pose-estimation folder.

$ python run.py --model=mobilenet_thin --resize=432x368 --image=./images/p1.jpg

Troubleshooting

Error: No module named ‘_pafprocess’ , you need to build c++ library for pafprocess– The error is self explanatory. It means we need to build pafprocess first. That can be done as shown below.

$conda install swig
$cd tf_pose/pafprocess

$ swig -python -c++ pafprocess.i && python3 setup.py build_ext --inplace
OR the following if the above does not work, try the following.
swig -python -c++ pafprocess.i && python setup.py build_ext --inplace

Error: No module named ‘tensorflow.contrib.tensorrt’

To get rid of this error, I had to comment a line within the estimator.py file. import tensorflow.contrib.tensorrt as trt)

Other things to check

• NVIDIA driver not up to date :
• git not installed
• Tensorflow version >=1.14.0
• swig not installed

References

• (Human Pose Estimation general summary) https://github.com/derikon/awesome-human-motion#introduction
• (OpenPose github repository) https://github.com/CMU-Perceptual-Computing-Lab/openpose
• (Understanding the theory) https://medium.com/analytics-vidhya/understanding-openpose-with-code-reference-part-1-b515ba0bbc73
• (OpenPose implementation using Tensorflow: tf-pose-estimation) https://github.com/ildoonet/tf-pose-estimation
• (How to install NVIDIA driver on Windows 10) https://www.top-password.com/blog/update-nvidia-display-driver-in-windows-10/
• (Install Tensorflow-GPU) https://www.youtube.com/watch?v=tPq6NIboLSc
• (Install OpenPose on Windows 10 tutorial) https://www.youtube.com/watch?v=nUjGLjOmF7o
• (OpenPose vs WrnchAI comparison) https://www.learnopencv.com/pose-detection-comparison-wrnchai-vs-openpose/#:~:text=It%20is%20clear%20that%20wrnchAI,for%20medium%20to%20large%20images.
• (AlphaPose github repository) https://github.com/MVIG-SJTU/AlphaPose
• Real-time human pose estimation in browser using TensorFlow.js) https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
• (Tutorial using OpenPose with TensorFlow 2.0x) https://towardsdatascience.com/realtime-multiple-person-2d-pose-estimation-using-tensorflow2-x-93e4c156d45f
• (Tutorial using Tensorflow 2.0) https://medium.com/@gsethi2409/pose-estimation-with-tensorflow-2-0-a51162c095ba
• (Cool Video) https://www.youtube.com/watch?v=RHRQoaqQIgo
• (Unofficial Windows Binaries for Python Extension Packages) https://www.lfd.uci.edu/~gohlke/pythonlibs/
• (PoseNet) https://www.youtube.com/watch?v=9KqNk5keyCc