river is a collection of lightweight services tailored for the efficient processing of video streams frame by frame, enabling near real-time inference.
river-serve and river-zmq-proxy consume a h264 video stream via webrtc, splits the stream into images and sends each to a zmq socket for inference.
when processing a frame we offset the timestamp by a fixed amount of time. this offset creates a buffer period where we can begin to process the next frame before the previous frame has finished processing. this is a simple but effective way to reduce latency. additionally, the updated timestamps preserve the original timing of the frames so the reassembled video is in sync but with a slight delay 🙌
river-onnx is to be run on N number of machines; it consumes images from an inbound zmq socket, performs inference and sends the results to an outbound zmq socket.
practically there would be many instances of river-onnx, and one instance of river-serve and river-zmq-proxy.
river-zmq-proxy is the critical link between river-serve and river-onnx. It's designed for non-blocking connections, enabling multiple threads to feed data to various workers simultaneously. Notably, it employs the ROUTER/DEALER pattern, which smartly routes messages to the next free worker, enhancing efficiency.
| Service | Description |
|---|---|
| river-serve | A small server that supports peer to peer WebRTC H264 video streaming |
| river-onnx | A small server that supports inference of ONNX models |
| river-app | A simple web app that uses river-serve to stream video from a webcam and river-onnx to perform inference on each frame |
| river-zmq-proxy | A small standalone application that handles connections between N number of DEALER sockets and a N number of ROUTER sockets |
this terminal is the video stream splitter and proxy services
# build docker with core services
make build
# run docker with core services
make runthis terminal is the inference service
cargo run --release --bin river-onnxthis terminal is the web app
cd river-app
bun dev