rubypoddar / MagicHandFX-

MagicHandFX is real-time hand gesture recognition project using OpenCV and MediaPipe. It detects hand gestures via webcam and overlays magical effects on detected gestures. The project enhances the user experience with responsive animations, featuring various magical effects based on different hand gestures, providing an engaging visual display.

MagicHandFX

MagicHandFX is an interactive real-time hand gesture recognition project using OpenCV and MediaPipe. It detects hand gestures via webcam and overlays magical effects on detected gestures. The project enhances the user experience with responsive animations, featuring various magical effects based on different hand gestures, providing an engaging visual display.

Detailed Description

MagicHandFX utilizes the powerful combination of OpenCV and MediaPipe to detect and track hand gestures in real-time via a webcam feed. The system processes each video frame, converting it to RGB and feeding it into the MediaPipe hands solution. This framework identifies hand landmarks and determines the presence of specific gestures based on landmark positions. When a gesture such as a clenched fist or open hand is recognized, a corresponding magical effect image is overlaid on the detected hand region, creating a visually stunning interactive experience. Each gesture triggers a different effect, and the effects dynamically change based on the hand's movements, making the interaction seamless and responsive.

---

Step-by-Step Explanation:

1. Import Libraries:

   import cv2
   import mediapipe as mp
   import numpy as np
   import os

   • cv2: OpenCV library for computer vision tasks.
   • mediapipe: Provides solutions for various tasks, including hand tracking.
   • numpy: Library for numerical operations.
   • os: Library for interacting with the operating system.

2. Initialize MediaPipe Hands:

   mp_hands = mp.solutions.hands
   hands = mp_hands.Hands(min_detection_confidence=0.8, min_tracking_confidence=0.8)

   • Sets up the MediaPipe Hands model with specified confidence thresholds for detection and tracking.

3. Define Paths to Gesture Images:

   base_path = 'Path Where All Gesture Images Located'
   image_paths = [
       os.path.join(base_path, 'image 1'),
       os.path.join(base_path, 'image 2'),
       os.path.join(base_path, 'image 3'),
       os.path.join(base_path, 'image 4')
   ]

   • Defines paths to images that will be overlaid based on detected gestures.

4. Load and Resize Gesture Images:

   gesture_images = []
   fixed_size = (300, 300)  # Fixed size for all images
   
   for path in image_paths:
       try:
           img = cv2.imread(path, cv2.IMREAD_UNCHANGED)
           if img is None:
               raise FileNotFoundError(f"Failed to load image: {path}")
           img = cv2.resize(img, fixed_size, interpolation=cv2.INTER_AREA)
           gesture_images.append(img)
       except Exception as e:
           print(f"Error loading image {path}: {e}")

   • Loads images from specified paths, resizes them to a fixed size, and stores them in gesture_images.

5. Video Capture:

   cap = cv2.VideoCapture(0)

   • Opens the default camera (index 0).

6. Overlay Transparent Function:

   def overlay_transparent(background, overlay, x, y):
       # Function to overlay a transparent image onto a background image.
       # Uses alpha blending to merge the overlay image with the background.

   • This function overlays a transparent image onto a background image at specified coordinates (x, y).

7. Gesture Detection Function:

   def is_fist(hand_landmarks, width, height):
       # Function to determine if the hand gesture is a fist based on landmark positions.

   • Checks if the hand gesture resembles a fist based on the detected landmarks.

8. Main Loop for Video Processing:

   while cap.isOpened():
       success, frame = cap.read()
       if not success:
           break

   • Reads frames from the camera until the user quits ('q' key).

9. Image Processing within the Loop:

       image_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
       result = hands.process(image_rgb)
   
       if result.multi_hand_landmarks:
           for hand_landmarks in result.multi_hand_landmarks:
               # Extracts bounding box coordinates of the detected hand.
               hand_x_min = int(min([lm.x * width for lm in hand_landmarks.landmark]))
               hand_y_min = int(min([lm.y * height for lm in hand_landmarks.landmark]))
               hand_x_max = int(max([lm.x * width for lm in hand_landmarks.landmark]))
               hand_y_max = int(max([lm.y * height for lm in hand_landmarks.landmark]))
   
               if is_fist(hand_landmarks, width, height):
                   if not gesture_detected:
                       gesture_index = (gesture_index + 1) % len(gesture_images)
                       gesture_detected = True
               else:
                   gesture_detected = False
   
               # Overlay gesture image on the frame based on the detected gesture.
               frame = overlay_transparent(frame, gesture_images[gesture_index], hand_x_min, hand_y_min)

   • Converts the frame to RGB format (required by MediaPipe), processes the frame using the hands model, and detects multiple hand landmarks.
   • Determines if the detected gesture is a fist and switches to the next gesture image (gesture_index) if a fist is detected.
   • Overlays the corresponding gesture image on the frame using the overlay_transparent function.

10. Display and User Interaction:

       cv2.imshow('Gesture Effect', frame)
    
       if cv2.waitKey(1) & 0xFF == ord('q'):
           break

- Displays the processed frame with overlaid gesture image.
- Checks for user input (`'q'` key) to exit the application.

11. **Release Resources:**
 ```python
cap.release()
cv2.destroyAllWindows()

• Releases the camera and closes all OpenCV windows.

graph TD;
    A[Start] --> B[Capture Video]
    B --> C[Flip Frame Horizontally]
    C --> D[Convert to RGB]
    D --> E[Detect Hands]
    E --> F[Check for Fist Gesture]
    F -- Yes --> G[Change Gesture Image]
    G --> H[Overlay Image]
    H --> I[Display Frame with Effect]
    F -- No --> I
    I --> J[Repeat]
    J --> B
    J -- Stop --> K[Release Resources]
    K --> L[End]

Loading