Gstreamer OpenCv backend. Error pushing buffer to Gstreamer pipeline
JuanDYB opened this issue · comments
Which version are you using?
v1.7.0
Which operating system are you using?
- Linux amd64 standard
- Linux amd64 Docker
- Linux arm64 standard
- Linux arm64 Docker
- Linux arm7 standard
- Linux arm7 Docker
- Linux arm6 standard
- Linux arm6 Docker
- Windows amd64 standard
- Windows amd64 Docker (WSL backend)
- macOS amd64 standard
- macOS amd64 Docker
- Other (please describe)
Describe the issue
I'm trying to create a live streaming using OpenCv to process images from camera. For that I'm using Gstreamer backend of OpenCv. I have tried to push frames using rtspclientsink in gstreamer pipeline and it fails pushing frames.
I have readed this issue and also tried with protocols=tcp.
Describe how to replicate the issue
- Build OpenCv with gstreamer
- Configure mediamtx configuration
yml - Copy example from docs
- Start server
- Start Python example from docs.
Server configuration yml
I'm using provided configuration in release download. I have only modified paths section.
paths:
mystream:
source: rtsp://mystream
OpenCv Build Information
General configuration for OpenCV 4.9.0 =====================================
Version control: unknown
Platform:
Timestamp: 2024-04-17T06:33:37Z
Host: Windows 10.0.19044 AMD64
CMake: 3.29.1
CMake generator: Visual Studio 17 2022
CMake build tool: C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe
MSVC: 1939
Configuration: Debug Release
CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (18 files): + SSSE3 SSE4_1
SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (9 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (38 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
AVX512_SKX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe (ver 19.39.33523.0)
C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /O2 /Ob2 /DNDEBUG
C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /Zi /Ob0 /Od /RTC1
C Compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP /O2 /Ob2 /DNDEBUG
C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP /Zi /Ob0 /Od /RTC1
Linker flags (Release): /machine:x64 /INCREMENTAL:NO
Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
ccache: NO
Precompiled headers: YES
Extra dependencies:
3rdparty dependencies:
OpenCV modules:
To be built: calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching ts video videoio
Disabled: world
Disabled by dependency: -
Unavailable: java python2
Applications: tests perf_tests apps
Documentation: NO
Non-free algorithms: NO
Windows RT support: NO
GUI: WIN32UI
Win32 UI: YES
VTK support: NO
Media I/O:
ZLib: build (ver 1.3)
JPEG: build-libjpeg-turbo (ver 2.1.3-62)
SIMD Support Request: YES
SIMD Support: NO
WEBP: build (ver encoder: 0x020f)
PNG: build (ver 1.6.37)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.5.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: YES (4.0.0)
GStreamer: YES (1.24.2)
DirectShow: YES
Media Foundation: YES
DXVA: YES
Parallel framework: Concurrency
Trace: YES (with Intel ITT)
Other third-party libraries:
Intel IPP: 2021.11.0 [2021.11.0]
at: C:/CMake/opencv/opencv/build/3rdparty/ippicv/ippicv_win/icv
Intel IPP IW: sources (2021.11.0)
at: C:/CMake/opencv/opencv/build/3rdparty/ippicv/ippicv_win/iw
Lapack: NO
Eigen: YES (ver 3.4.0)
Custom HAL: NO
Protobuf: build (3.19.1)
Flatbuffers: builtin/3rdparty (23.5.9)
OpenCL: YES (NVD3D11)
Include path: C:/CMake/opencv/opencv/3rdparty/include/opencl/1.2
Link libraries: Dynamic load
Python 3:
Interpreter: C:/Program Files/Python312/python.exe (ver 3.12.2)
Libraries: C:/Program Files/Python312/libs/python312.lib (ver 3.12.2)
numpy: C:/Users/JuanDYB/AppData/Roaming/Python/Python312/site-packages/numpy/core/include (ver 1.26.4)
install path: C:/Program Files/Python312/Lib/site-packages/cv2/python-3.12
Python (for build): C:/Program Files/Python312/python.exe
Java:
ant: NO
Java: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Install to: C:/CMake/opencv/opencv/build/install
-----------------------------------------------------------------
Python example
import os
os.add_dll_directory("C:\\gstreamer\\1.0\\msvc_x86_64\\bin")
from datetime import datetime
from time import sleep, time
import cv2
import numpy as np
fps = 15
width = 800
height = 600
colors = [
(0, 0, 255),
(255, 0, 0),
(0, 255, 0),
]
out = cv2.VideoWriter('appsrc ! videoconvert' + \
' ! video/x-raw,format=I420' + \
' ! x264enc speed-preset=ultrafast bitrate=600 key-int-max=' + str(fps * 2) + \
' ! video/x-h264,profile=baseline' + \
' ! rtspclientsink protocols=tcp location=rtsp://localhost:8554/mystream',
cv2.CAP_GSTREAMER, 0, fps, (width, height), True)
if not out.isOpened():
raise Exception("can't open video writer")
curcolor = 0
start = time()
while True:
frame = np.zeros((height, width, 3), np.uint8)
# create a rectangle
color = colors[curcolor]
curcolor += 1
curcolor %= len(colors)
for y in range(0, int(frame.shape[0] / 2)):
for x in range(0, int(frame.shape[1] / 2)):
frame[y][x] = color
out.write(frame)
print("%s frame written to the server" % datetime.now())
now = time()
diff = (1 / fps) - now - start
if diff > 0:
sleep(diff)
start = now
Did you attach the server logs?
yes
OpenCv Python output
2024-04-17 09:33:23.429206 frame written to the server
2024-04-17 09:33:23.538309 frame written to the server
2024-04-17 09:33:23.651449 frame written to the server
2024-04-17 09:33:23.746444 frame written to the server
2024-04-17 09:33:23.858790 frame written to the server
2024-04-17 09:33:23.987974 frame written to the server
2024-04-17 09:33:24.091015 frame written to the server
[ WARN:0@1.780] global cap_gstreamer.cpp:2839 cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module rtspclientsink0 reported: Could not read from resource.
[ WARN:0@1.793] global cap_gstreamer.cpp:2730 cv::CvVideoWriter_GStreamer::writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
2024-04-17 09:33:24.232207 frame written to the server
[ WARN:0@1.896] global cap_gstreamer.cpp:2730 cv::CvVideoWriter_GStreamer::writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
2024-04-17 09:33:24.343710 frame written to the server
[ WARN:0@2.004] global cap_gstreamer.cpp:2730 cv::CvVideoWriter_GStreamer::writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
2024-04-17 09:33:24.451317 frame written to the server
[ WARN:0@2.117] global cap_gstreamer.cpp:2730 cv::CvVideoWriter_GStreamer::writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
Media Server output
C:\Users\Usr\Downloads\mediamtx_v1.7.0_windows_amd64>mediamtx.exe
2024/04/17 09:33:12 INF MediaMTX v1.7.0
2024/04/17 09:33:12 INF configuration loaded from C:\Users\Usr\Downloads\mediamtx_v1.7.0_windows_amd64\mediamtx.yml
2024/04/17 09:33:12 INF [path mystream] [RTSP source] started
2024/04/17 09:33:12 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
2024/04/17 09:33:12 INF [RTMP] listener opened on :1935
2024/04/17 09:33:12 INF [HLS] listener opened on :8888
2024/04/17 09:33:12 INF [WebRTC] listener opened on :8889 (HTTP), :8189 (ICE/UDP)
2024/04/17 09:33:12 INF [SRT] listener opened on :8890 (UDP)
2024/04/17 09:33:15 ERR [path mystream] [RTSP source] dial tcp: lookup mystream: no such host
2024/04/17 09:33:22 ERR [path mystream] [RTSP source] dial tcp: lookup mystream: no such host
2024/04/17 09:33:23 INF [RTSP] [conn [::1]:50391] opened
2024/04/17 09:33:24 INF [RTSP] [session 2ffa1e01] created by [::1]:50391
2024/04/17 09:33:24 INF [RTSP] [conn [::1]:50391] closed: can't publish to path 'mystream' since 'source' is not 'publisher'
2024/04/17 09:33:24 INF [RTSP] [session 2ffa1e01] destroyed: not in use
2024/04/17 09:33:24 INF [RTSP] [conn [::1]:50393] opened
2024/04/17 09:33:24 INF [RTSP] [conn [::1]:50393] closed: EOF
2024/04/17 09:33:29 ERR [path mystream] [RTSP source] dial tcp: lookup mystream: no such host
2024/04/17 09:33:37 ERR [path mystream] [RTSP source] dial tcp: lookup mystream: no such host
2024/04/17 09:33:44 ERR [path mystream] [RTSP source] dial tcp: lookup mystream: no such host
2024/04/17 09:33:51 ERR [path mystream] [RTSP source] dial tcp: lookup mystream: no such host
2024/04/17 09:33:55 INF shutting down gracefully
2024/04/17 09:33:55 INF [SRT] listener is closing
2024/04/17 09:33:55 INF [WebRTC] listener is closing
2024/04/17 09:33:55 INF [HLS] listener is closing
2024/04/17 09:33:55 INF [RTMP] listener is closing
2024/04/17 09:33:55 INF [RTSP] listener is closing
2024/04/17 09:33:55 INF [path mystream] [RTSP source] stopped: path is closing
2024/04/17 09:33:55 INF waiting for running hooks
Did you attach a network dump?
no
Your mediamtx configuration assumes that you're running an RTSP server on GStreamer, but that's not the case since you're using rtspclientsink instead of building on top of gst-rtsp-server.
Remove the line "source: rtsp://mystream" from your mediamtx configuration file, and GStreamer will then publish the stream to that path assuming the rest of your code works properly. If unsure, test the setup on command line with something like:
gst-launch-1.0 videotestsrc ! x264enc ! rtspclientsink protocols=tcp location=rtsp://localhost:8554/mystream
You are using MediaMTX to pull a stream from a RTSP server, while an OpenCV script is a RTSP client. The process is different. The configuration that you need is just the default one, or alternatively, if you want to write it explicitly:
paths:
mystream:
source: publisherHi,
I'm new to Gstreamer. I wanted to stream as RTSP live streaming from camera frames and for that I'm using Gstreamer and OpenCv, the problem is that Gstreamer only publish an RTP stream which it's not possible to see from multiple clients. That the reason to use RTSP server.
So using this configuration Gstreamer will pusblish over MediaMtx server the RTSP stream?
What will be the best configuration for server to reduce as less as possible the latency? I want to reduce it because I'm publishing live streaming.
With regards,