StreamHttpResponse with sync operations break in Daphne

[Dipan Nanda]

I am basically using this

def process(
    self,
    request: Request,
    pk: str,
    <parameters>
):
    video = Video.objects.get(id=pk)
    return StreamingHttpResponse(process_video(<parameters>))

As you can tell process_video is a sync function but the generator breaks for this and it only yields the final result. This is working fine with --no-asgi flag.

To give a example of how it looks like (a much smaller version of the code). But it processed video and send the frame to the frontend.

def process_video(
        <parameters>
):
    # Open the video file
    cap = cv2.VideoCapture(video_path)

    while cap.isOpened():
                  success, frame = cap.read()
                  print (frame)
                  yeild (frame)
    yield ("Complete")

It is basically a sync generator, but it fails when daphne is introducted. It only yeilds "Complete" once complete. The print function is printing the frame perfectly fine. So I know it runs. I also tried to convert it to async but async generator is a error.

[Dipan Nanda]

Issue References

1. #413

I initially has a async_to_sync bug which got fixed by updated django to the latest lts version (4.2.5). But the generator still seems to be a issue. I have faced caroutine bug while trying to tackle this as well.

[Carlton Gibson]

Yes, you need to use Django 4.2's support for async streaming responses (using an async generator).

[Dipan Nanda]

I have converted to code to async generator and now it works fine. But I think we should have some way to convert a sync generator to async generator or add support for sync generator in StreamHTTPResponse.

[Dipan Nanda]

I am closing this issue for now but I think we should look into one of those things I mentioned above.

[Carlton Gibson]

@themagicalmammal Glad you've resolved your issue.

I'm afraid there's no magic way to convert sync iterators to async iterators. As per the Django docs on this topic, you need to adapt the generator passed to your response depending on whether you're running under WSGI or ASGI. (Alas the abstraction breaks down at this point.)