bufpool
bufpool is an implementation of a pool of byte buffers with anti-memory-waste protection. It is based on the code and ideas from these 2 projects:
bufpool consists of global pool of buffers that have a capacity of a power of 2 starting from 64 bytes to 32 megabytes. It also provides individual pools that maintain usage stats to provide buffers of the size that satisfies 95% of the calls. Global pool is used to reuse buffers between different parts of the app.
Installation
go get github.com/vmihailenco/bufpool
Usage
bufpool can be used as a replacement for sync.Pool
:
var jsonPool bufpool.Pool // basically sync.Pool with usage stats
func writeJSON(w io.Writer, obj interface{}) error {
buf := jsonPool.Get()
defer jsonPool.Put(buf)
if err := json.NewEncoder(buf).Encode(obj); err != nil {
return err
}
_, err := w.Write(buf.Bytes())
return err
}
or to allocate buffer of the given size:
func writeHex(w io.Writer, data []byte) error {
n := hex.EncodedLen(len(data)))
buf := bufpool.Get(n) // buf.Len() is guaranteed to equal n
defer bufpool.Put(buf)
tmp := buf.Bytes()
hex.Encode(tmp, data)
_, err := w.Write(tmp)
return err
}
If you need to append data to the buffer you can use following pattern:
buf := bufpool.Get(n)
defer bufpool.Put(buf)
bb := buf.Bytes()[:0]
bb = append(bb, ...)
buf.ResetBytes(bb)
You can also change default pool thresholds:
var jsonPool = bufpool.Pool{
ServePctile: 0.95, // serve p95 buffers
}