forwardcache

A distributed forward caching proxy for Go's http.Client using httpcache and heavily inspired by groupcache. Backed by a lot of existing cache backends thanks to httpcache. A per host LRU algorithm is provided to optionally front any existing cache. Like groupcache, forwardcache "is a client library as well as a server. It connects to its own peers."

In simple terms, it is a distributed cache for HEAD and GET requests. It follows the HTTP RFC so it will only cache cacheable responses (like browsers do).

Docs on godoc.org

+---------------------------------+
|                                 |
|         origin servers          |
|                                 |
+--+-------+-------+-----------+--+
   ^       ^       ^           ^
   |       |       |           |
   |       |       |           |
+-----+ +-----+ +-----+     +-----+
|cache| |cache| |cache|     |cache|
+--+--+ +--+--+ +--+--+     +--+--+
|     | |     | |     |     |     |
|  1  +-+  2  +-+  3  +--|--+  N  |
|     | |     | |     |     |     |
+-----+ +-----+ +-----+     +-----+

Requirements

Go 1.7 (using request's context)

Motivation

Needed requests to be cached
Needed bandwidth to be spread among N peers
Needed to cache HTTPS requests
Needed a lightweight infrastructure

So I couldn't go with existing solutions like Apache Traffic Server or Squid.

Process

When making a request through the pool...

the requested url is hashed to determine which peer is responisble to proxy the request to the origin
the peer is called, if the request is cached and valid, it is returned without contacting the origin
if the request is not cached, the request is fetched from the origin and cached if cacheable before being returned back to the client

Example

peer := NewPeer("http://10.0.1.1:3000")
peer.SetPool("http://10.0.1.1:3000", "http://10.0.1.2:3000")

// -or-

client := NewClient(WithPool("http://10.0.1.1:3000", "http://10.0.1.2:3000"))
peer = NewPeer("http://10.0.1.1:3000", WithClient(client))

// -then-

http.DefaultTransport = peer
http.Get("https://...js/1.5.7/angular.min.js")

// -or-

http.DefaultClient = peer.HTTPClient()
http.Get("https://...js/1.5.7/angular.min.js")

// -or-

c := peer.HTTPClient()
c.Get("https://...js/1.5.7/angular.min.js")

// ...

http.ListenAndServe(":3000", peer.Handler())

Licence

Apache 2.0 (see LICENSE and NOTICE)

mikegleasonjr / forwardcache