ArshiaS7 / Hand-Tracking

Hand-Tracking Project with Python: Using OpenCV & MediaPipe

Hand-Tracking

Hand-Tracking Project with Python: Using OpenCV & MediaPipe

Explore the docs »

View Demo . Report Bug . Request Feature

Table of Contents

1. About The Project
2. Getting Started
3. Coding
   • Step 1 - Importations and initializations
   • Step 2 - Image Capturing and Processing
   • Step 3 - Working with Each Hand

About The Project

Hand tracking is the process in which a computer uses computer vision to detect a hand from an input image and keeps focus on the hand’s movement and orientation. Hand tracking allows us to develop numerous programs that use hand movement and orientation as their input.

Getting Started

To create the program that will perform hand tracking, we will need two Python libraries. These are openCV and MediaPipe.

We will use openCV to perform operations associated with computer vision. We will use MediaPipe to perform the actual hand detection and tracking on our input image.

• OpenCV

To install openCV, use the following command:

pip install opencv-python

• MediaPipe

Type the following command in the terminal to install MediaPipe:

pip install mediapipe

Hand-Tracking Program

Hand tracking using MediaPipe involves two stages:

• Palm detection: MediaPipe works on the complete input image and provides a cropped image of the hand.
• Hand landmarks identification: MediaPipe finds the 21 hand landmarks on the cropped image of the hand.

The 21 hand points that MediaPipe identifies are shown in the image below:

Coding

Step 1 - Importations and initializations

We start by importing the two libraries we discussed. Importing the libraries enables us to use its dependencies.

We will then create an object cap for video capturing. We require the other three objects to manipulate our input using MediaPipe:

import cv2
import mediapipe as mp

cap = cv2.VideoCapture(0)
mpHands = mp.solutions.hands
hands = mpHands.Hands()
mpDraw = mp.solutions.drawing_utils

Step 2 - Image Capturing and Processing

The code below takes the image input from the webcam. It then converts the image from BGR to RGB. This is because MediaPipe only works with RGB images, not BGR.

It then processes the RGB image to identify the hands in the image:

while True:
    success, image = cap.read()
    imageRGB = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = hands.process(imageRGB)

Step 3 - Working with Each Hand

    if results.multi_hand_landmarks:
        for handLms in results.multi_hand_landmarks:
            mpDraw.draw_landmarks(img, handLms, mpHands.HAND_CONNECTIONS)

    cv2.imshow("Image", img)

    if cv2.waitKey(1) == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

In the code above, we use the if statement to check whether a hand is detected. We then use the first for loop to enable us work with one hand at a time.

The second for loop helps us get the hand landmark information which will give us the x and y coordinates of each listed point in the hand landmark diagram.

Hand-Tracking:

Hand_Tracking.mp4