maxim2266 / cache

Another generic cache for Go.

Generic LRU cache for Go

Probably, a 121^st implementation of cache since Go generics were introduced, but this one actually solves the race condition problem that exists in some other more starred packages.

API

An LRU cache for any given types K and V ("key" and "value", respectively) can be constructed using function

func New(size int, ttl time.Duration, backend func(K) (V, error)) *Cache[K,V]

Parameters:

• Maximum size of the cache (a positive integer);
• Time-to-live for cache elements (can be set to something like ten years to retain "forever");
• Backend function to call when a cache miss occurs. The function is expected to return a value for the given key, or an error. Both the value and the error are stored in the cache. A slow backend function is not going to block access to the entire cache, only to the corresponding value.

The constructor returns a pointer to a newly created cache object.

A cache object has two public methods:

• Get(K) (V, error): given a key, it returns the corresponding value, or an error. On cache miss the result is transparently retrieved from the backend. The cache itself does not produce any error, so all the errors are from the backend only. Notably, this method has the same signature as the backend function, and it may be considered as a wrapper around the backend that adds memoisation. For example, given the backend function

   func getUserInfo(userID int) (*UserInfo, error)

  a caching wrapper with the same signature can be created like

   getUserInfoCached := cache.New(1000, 2 * time.Hour, getUserInfo).Get

  (assuming in this particular scenario there is no need to ever delete a record from the cache).
• Delete(K): deletes the specified key from the cache; no-op if the key is not present.

The cache object is safe for concurrent access. To flush the cache simply replace it with a newly created one.

Benchmarks

▶ go version
go version go1.22.0 linux/amd64
▶ go test -bench .
goos: linux
goarch: amd64
pkg: github.com/maxim2266/cache
cpu: Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz
BenchmarkCache-6            	19329848	        60.20 ns/op
BenchmarkContendedCache-6   	  873910	      1697 ns/op

Here the first benchmark reads the cache from a single goroutine, while the second one is the same benchmark running in parallel with another 10 goroutines accessing the cache concurrently. The cache is instantiated with integer keys and values.