Modern cloud web services developed in Go execute on top of a runtime environment. On the one hand, Go runtime provide several off-the-shelf benefits like code security and cross-platform execution. On the other side, runtime's internal routines such as automatic memory management add a non-deterministic overhead to the overall service time, increasing the tail of the service time distribution. In this context, it is well known that the Garbage Collector is among the leading causes of high tail latency.
To tackle this problem, we have developed the Garbage Collector Control Interceptor (GCI) -- a request interceptor agnostic regarding the cloud service the load it is subjected load to. GCI helps to improve the service time of cloud services by controlling GC interventions and using simple load shedding mechanisms to signal load balancers or other clients, preventing serving requests during these interventions.
- Message Push benchmark description and results can be found here.
- GCI-go on the cloud:
go get -u github.com/gcinterceptor/gci-go/...Let's say you you're building your cloud service using the Go's net/http package. To start using GCI simply wrap your service endpoint with httphandler.GCI. For example, imagine your have a variable hello, which points to your endpoint http.HandlerFunc:
http.Handle("/", httphandler.GCI(hello))A complete example here.
Would to have GCI on your favourite framework? Please send us a PR or open an issue.
2017
Using Load Shedding to Fight Tail-Latency on Runtime-Based Services. Fireman, D.; Lopes, R; Brunet, J. XXIX Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos (SBRC).