[BUG]: Intermittent Packet Type Mismatch Error in Long-Term Face Detection with DeepFace and MediaPipe "Graph has errors: Packet type mismatch on a calculator receiving from stream "image": ; Empty packets are not allowed for type: OneOf<::mediapipe::Image, ::mediapipe::ImageFrame, mediapipe::GpuBuffer>"

[Mathews Edwirds]

Before You Report a Bug, Please Confirm You Have Done The Following...

• I have updated to the latest version of the packages.
• I have searched for both existing issues and closed issues and found none that matched my issue.

DeepFace's version

v0.0.91

Python version

3.10

Operating System

Windows 10 and inside docker 22.04

Dependencies

opencv_python>=4.6.0.66
opencv_contrib_python>=4.6.0.66
tf-keras
#tensorflow>=2.8.0
tensorflow[and-cuda]>=2.8.0
deepface>=0.0.79
mtcnn>=0.1.1
mediapipe>=0.10.7
retina-face>=0.0.13
torch>=1.13.1
torchvision>=0.15.1+cu118
ultralytics>=8.0.96
protobuf>=3.20.3
boto3>=1.26.143
blinker>=1.6.2
imutils>=0.5.4
numpy>=1.23.5
pika>=1.3.2
Pillow>=10.1.0
python-socketio>=5.10.0
pytz>=2023.3
requests>=2.31.0
python-socketio[client] #websocket-client package not installed only polling transport is available

Reproducible example

Here is a snippet of the code that manages the camera and face detection:


import cv2
import numpy as np
from deepface import DeepFace

def IsValidFrame(frame):
    if frame is None or isinstance(frame, (list, tuple)):
        return False
    if isinstance(frame, np.ndarray):
        try:
            return frame.shape[0] > 50 and frame.shape[1] > 50
        except:
            return False
    return False

USE_IMUTILS = True
backends = ['opencv', 'ssd', 'dlib', 'mtcnn', 'retinaface', 'mediapipe']
DETECTOR_CAM = ParseCameras()

while True:
    try:
        frame = DETECTOR_CAM["cap"].read() if USE_IMUTILS else DETECTOR_CAM["cap"].read()[1]
    except Exception as e:
        print(f'[Detector] Error reading frame from camera "{DETECTOR_CAM["source"]}": {e}')

    if not IsValidFrame(frame):
        HandleEmptyFrame(process_type, DETECTOR_CAM)
    else:
        frame = cv2.resize(frame, DETECTOR_CAM["streaming_resolution"])
        try:
            face_objs = DeepFace.extract_faces(img_path=frame, enforce_detection=False, detector_backend=backends[-1])
        except Exception as ex:
            print(f"[Detector] Face detection error: The frame may be empty or no face detected. Error: {ex}")

Relevant Log Output

Graph has errors: Packet type mismatch on a calculator receiving from stream "image": ; Empty packets are not allowed for type: OneOf<::mediapipe::Image, ::mediapipe::ImageFrame, mediapipe::GpuBuffer>

Expected Result

No response

What happened instead?

No response

Additional Info

Firstly, I want to extend my gratitude for developing such an impressive library like DeepFace. It has been instrumental in my personal project, which involves face detection through an IP camera streaming setup with the RTSP protocol.

I am running my code inside a Docker container. It includes proper methods for connecting and reconnecting to the camera in case of disconnection, and validates frames before processing them with DeepFace using the "mediapipe" backend. However, after approximately 48 hours of continuous processing, I encounter the following error, and the process only resumes normally after restarting the container:

Graph has errors: Packet type mismatch on a calculator receiving from stream "image": ; Empty packets are not allowed for type: OneOf<::mediapipe::Image, ::mediapipe::ImageFrame, mediapipe::GpuBuffer>

Could you please provide some insights or suggestions on how to handle or prevent this error? Any guidance on how to stabilize the long-term processing or understanding why this error occurs would be greatly appreciated.

[Sefik Ilkin Serengil]

Never seen this error before but seems mediapipe throws it. You may consider to raise an issue in mediapipe's repo.

[Sefik Ilkin Serengil]

closed due to inactivity