dusty-nv / jetson-utils

C++/CUDA/Python multimedia utilities for NVIDIA Jetson

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

VideoOutput.Render() to webrtc failure using appsink frame

Hongfeng-mtec opened this issue · comments

I am trying to use appsink to get a video frame, which I convert to cudaImage, then try to stream it to webrtc.
The code is largely like this:

#pipeline: ... ! videoconvert ! video/x-raw,format=RGB ! appsink

self.output = videoOutput('webrtc://@:8554/live')

def __on_new_sample(self,app_sink):
        sample = app_sink.emit("pull-sample") 
        caps = sample.get_caps()
        height = caps.get_structure(0).get_value("height")
        width = caps.get_structure(0).get_value("width")
        buffer = sample.get_buffer()
        numpy_frame = np.ndarray(
            shape=(height, width, 3),
            dtype=np.uint8,
            buffer=buffer.extract_dup(0, buffer.get_size()))
        cuda_img= cudaFromNumpy(numpy_frame)
        print(cuda_img)
        self.output.Render(cuda_img)

Now the cudaImage object looks legit, but I am get following error on the render function.

<cudaImage object>
   -- ptr:      0x102360000
   -- size:     2764800
   -- width:    1280
   -- height:   720
   -- channels: 3
   -- format:   rgb8
   -- mapped:   true
   -- freeOnDelete: true
   -- timestamp:    0.000000

TypeError: can't convert return value to desired type

If I output the video to v4l2sink, then run video_viewer using input from the loopback device, it works fine.
L4T: 32.7.4, Hardware: TX2

another data point: if I just modify python/examples/cuda_from_numpy.py, and append

output = videoOutput('webrtc://@:8555/live')  
while True:
    output.Render(cuda_mem)
    time.sleep(0.1)

that script works fine, I am wondering what's the difference here.

it turns out nothing related to videoOutput Render(), if I add "return 0" to the new-sample callback, the error message goes away. sorry for the false alarm.