naphthalene / esp32_fpv_rc

low latency camera streaming with UDP for esp32-camera.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Low Latency Camera Streaming with UDP for ESP32-Camera

Targetting M5 Stack Unit Cam S3, should work on any S3 with OV2640. Requires PSRAM. M8 8Servo controller supported! Modules can be run separately! you can use camera and no servos, or servos and no camera.

COMING SOON:

servo control with direct gpio
vr/ar viewer for racing rig

Samples:

CIF 60p

SVGA 30p

HD 15p

esp32_sender

Tested with ESP-IDF 5.4.

All settings can be configured through menuconfig.
Supports Logging to UDP, can receive with nc -u -l 55557 if ESP_UDP_LOG is enabled

WiFi

The SSID and Network key must be set before compiling. You can set it with IDF or from the config directly

cd esp32_sender
idf.py menuconfig

in sdkconfig:

CONFIG_ESP_WIFI_SSID="MyNetwork"
CONFIG_ESP_WIFI_PASSWORD="MyKey"

Building for ESP32

Requires ESP-IDF and esp32_camera fragment mode: https://github.com/arms22/esp32-camera-fragment-mode

cd esp32_sender
idf.py -p PORT build flash monitor

python_receiver

Installing requirements

pip3 install -r ./python_receiver/requirements.txt

Running the app

The easiest way to conenct is with the scanning and launching script 'main.py'. This file will broadcast for any running cameras and list the IP addresses. You can select 1-2 cameras and pass them to the QT or GL viewers.

python3 python_receiver/main.py

Yuu can also launch a viewer directly by specifying the addresses

python3.exe python_receiver/qtcontroller.py <esp32_ip>

For the VR/OpenGL viewer: (work in progress)

python3.exe python_receiver/vrcontroller.py <esp32_ip_1> <esp32_ip_2:optional>

Project Status

esp32_sender

Working

  • Send camera stream with very low latency.
  • Redirect logs to UDP

Issues

  • Servo Control not fully implemented

QT Viewer

Working

  • receives low latency camera

Issues

  • does not send control info

GL Viewer

Working

  • receives low latency camera
  • receives second camera

Issues

  • second camera not drawing fast enough
  • coded like a wild monkey did it
  • does not send control info

About

low latency camera streaming with UDP for esp32-camera.


Languages

Language:C 53.2%Language:Python 42.9%Language:C++ 2.0%Language:Makefile 1.0%Language:CMake 0.9%