ERROR tcam-libtcam timeout while waiting for new image buffer
marcoabrate opened this issue · comments
Describe the bug
I am trying to record videos from a DMK 37AUX273 camera on a Raspberry Pi 4 model B, using Python. I am facing an issue with a recurring warning and a recurring error, both distruptive the recording by either shortening, corrupting or stopping the recording. The warning is:
0:05:02.151105863 3009 0xf0606180 WARN tcam-libtcam V4l2Device.cpp:1266:stream: Did not receive image for long time.
and the error:
0:00:56.567113482 3349 0xeff06580 ERROR tcam-libtcam V4l2Device.cpp:1244:stream: Timeout while waiting for new image buffer.
To Reproduce
The warnings and errors occur multiple times with the scripts I am running. I am running three scripts to save videos as frames, AVI, and MP4 files. Here's the script that saves frames:
#!/usr/bin/env python3
import time
import sys
import gi
import os
from datetime import datetime
gi.require_version("Gst", "1.0")
from gi.repository import Gst
def main():
Gst.init(sys.argv)
Gst.debug_set_default_threshold(Gst.DebugLevel.WARNING)
# Set this to a serial string for a specific camera
serial = "31320403"
# To save frames as separate JPG images:
pipeline = Gst.parse_launch(
"tcambin name=source"
" ! video/x-raw,format=BGRx,width=1440,height=1080,framerate=15/1"
" ! videoconvert"
" ! jpegenc"
" ! multifilesink name=fsink"
)
suffix = datetime.now().strftime("%d%m%Y_%H%M%S")
save_dir = f"./saved_videos/saved_stream_{suffix}"
if not os.path.exists(save_dir):
os.makedirs(save_dir)
save_path = os.path.join(save_dir, "frame%06d.jpg")
fsink = pipeline.get_by_name("fsink")
fsink.set_property("location", save_path)
camera = pipeline.get_by_name("source")
if serial:
camera.set_property("serial", serial)
pipeline.set_state(Gst.State.PLAYING)
stream_time = 3600 # seconds
start_time = time.time()
print(f"[+] starting video stream for {int(stream_time/60)} minutes")
print("[*] or press Ctrl-C to stop")
try:
while True:
seconds = time.time() - start_time
if seconds%600 == 0 : print(f"[*] recorded for {int(seconds/60)} minutes")
if seconds > stream_time : break
except KeyboardInterrupt:
pass
finally:
print("[*] closing stream")
# cleanup, reset state
pipeline.set_state(Gst.State.NULL)
return 0
if __name__ == '__main__':
main()Example output that corrupts the recording (i.e. the number of frames saved is much smaller than seconds*FPS):
pi@raspberrypi:~/project $ python3 save_stream_frames.py
[+] starting video stream for 5 minutes
[*] or press Ctrl-C to stop
0:00:56.567113482 3349 0xeff06580 ERROR tcam-libtcam V4l2Device.cpp:1244:stream: Timeout while waiting for new image buffer.
0:02:38.637817130 3349 0xeff06580 ERROR tcam-libtcam V4l2Device.cpp:1244:stream: Timeout while waiting for new image buffer.
0:02:53.149090979 3349 0xeff06580 ERROR tcam-libtcam V4l2Device.cpp:1244:stream: Timeout while waiting for new image buffer.
0:04:33.191870729 3349 0xeff06580 ERROR tcam-libtcam V4l2Device.cpp:1244:stream: Timeout while waiting for new image buffer.
[*] closing stream
pi@raspberrypi:~/project $
The other two scripts are mostly the same, only the pipeline changes. For AVI
pipeline = Gst.parse_launch(
"tcambin name=source"
" ! video/x-raw,format=BGRx,width=1440,height=1080,framerate=15/1"
" ! videoconvert"
" ! avimux"
" ! filesink name=fsink"
)and for MP4:
pipeline = Gst.parse_launch(
"tcambin name=source"
" ! video/x-raw,format=BGRx,width=1440,height=1080,framerate=15/1"
" ! videoconvert"
" ! x264enc"
" ! mp4mux"
" ! filesink name=fsink"
)Expected behavior
The scripts should save the videos of the specified length without errors.
computer used (please complete the following information):
- OS: Raspbian GNU/Linux 11 (bullseye)
- Architecture/Platform: Raspberry Pi 4 model B
- Camera model you use: DMK 37AUX273
- tiscamera version (tcam-ctrl --version, if possible):
Versions:
Tcam: 1.1.1_master/ee45fcf7_rev_4142
Aravis: 0.8_version_e977fa4
Modules: gst_aravis_arvu3v_v4l2_libusb_tools_doc_tests
thank you
Hello
The error message tells us, there are no more (complete) images received from the camera (obviously). This can happen, if the CPU load is too high or the bus in the PI is full. However, 15 fps at this resolution is not too much data, so there should be no issues with USB 3 connections. I suppose, you connected the camera to the USB 3 ports of the PI 4
Example output that corrupts the recording (i.e. the number of frames saved is much smaller than seconds*FPS):
That is expected, because the SD card is not fast enough for saving so many single images per second. There is a similar issue in Windows, when Windows flushes the disc-cache, no more write operations are possible
A first step for narrowing down the issue is to use the xvimagesink instead of saving the images:
pipeline = Gst.parse_launch(
"tcambin name=source"
" ! video/x-raw,format=BGRx,width=1440,height=1080,framerate=15/1"
" ! videoconvert"
" ! xvimagesink"
)If this works well, then the issue is at saving only. Which makes the things more complicated, because we have to read in the Gstreamer documentation about how to optimize the parameters for the used encoders. This is a pure software issue then. If I remember correctly, there were the omx* GStreamer modules for compression, which make a better use of the PI's hardware. You may try them.
Stefan
Hi @TIS-Stefan and thank you. I understand how the problem could be at the SD card level, and we did make sure to buy the fastest SD cards available to avoid bottlenecks in the saving process. I will try this xvimagesink.
However, this problem persists when saving with the AVI and MP4 formats, do you have any ideas why this might be the case?
Hi marcoabrate,
However, this problem persists when saving with the AVI and MP4 formats, do you have any ideas why this might be the case?
Currently not.
But i would like you to be sure about the connection between Raspi 4 and camera. Is the camera connected to a USB 3 Port?
Also what type of cable is used?
Some Cables with a Type C Connector especially for Smartphones might not work correctly with our cameras.
Also let us know your results with the xvimagesink
hi @TIS-Kevin, sorry for the late reply. The camera is connected to the Raspi 4 with the Amazon Basics usb-c to usb-3.0 cable.
running with the xvimagesink does not generate any errors and I can see the videos without any frame dropping or other issues. At this point the problem must be at saving time, I will check the omx* GStreamer modules for compression @TIS-Stefan suggested.
Hello
I guess, an SD card is too slow to save video video files with 1440x1024@15 fps. I never tried that on my own.
If I would need to save video files on a PI, I would try something like a RAM disc, save the video there first and copy it later to SD card.
Stefan