go-chunked-streaming-server

This is simple webserver written in GoLang that allows you to chunk transfer from the ingress and egress sides. In other words:

If it receives a GET request for object A, and object A is still being ingested, we send all we have of object A and we keep that GET request open sending on real time (and in memory) all the chunks that we received for object A, until that object is closed.
If we receives a GET request for object B and this object is already written (and closed), we send the object B as any other static web server.

Ideal for low latency media processing pipelines.

Usage

Installation

Just download GO in your computer. See GoLang
Create a Go directory to be used as a workspace for all go code, i.e.

mkdir ~/MYDIR/go

Add GOPATH to your ~/.bash_profile or equivalent for your shell

export GOPATH="$HOME/MYDIR/go"

Add GOPATH/bin to your path in ~/.bash_profile or equivalent for your shell

export PATH="$PATH:$GOPATH/bin

Restart your terminal or source your profile
Clone this repo:

go get github.com/mjneil/go-chunked-streaming-server

Go the the source code dir `

cd $HOME/MYDIR/go/src/github.com/mjneil/go-chunked-streaming-server

Compile main.go doing:

make

Testing

You can execute ./bin/./go-chunked-streaming-server -h to see all the possible command arguments.

Usage of ./bin/go-chunked-streaming-server:
  -c string
        Certificate file path (only for https)
  -d    Indicates to remove files from the server based on original Cache-Control (max-age) header
  -i int
        Port used for HTTP ingress/ egress (default 9094)
  -k string
        Key file path (only for https)
  -o string
        JSON file path with the CORS headers definition
  -p string
        Path used to store (default "./content")
  -r    Indicates DO NOT use disc as persistent/fallback storage (only RAM)

Example simple HTTP

Start the server

./bin/./go-chunked-streaming-server

Upload a file

curl http://localhost:9094 --upload-file file.test.txt

Consume the file and saved it to disc

curl http://localhost:9094/file.test.txt -o file.test.downloaded.txt

Example chunked HTTP

We could build a low latency HLS pipeline in conjunction with go-ts-segmenter by doing:

Start webserver

./bin/./go-chunked-streaming-server

Generate LHLS data

ffmpeg -f lavfi -re -i smptebars=size=320x200:rate=30 -f lavfi -i sine=frequency=1000:sample_rate=48000 -pix_fmt yuv420p -c:v libx264 -b:v 180k -g 60 -keyint_min 60 -profile:v baseline -preset veryfast -c:a aac -b:a 96k -f mpegts - | manifest-generator -l 3 -d 2

Play that data with any HLS player (recommended any player that implements low latency mode)

ffplay http://localhost:9094/results/chunklist.m3u8

Or use Safari with this URL http://localhost:9094/results/chunklist.m3u8

mjneil / go-chunked-streaming-server