Reduce GC overhead by using custom memory allocator
mangalaman93 opened this issue · comments
You will likely want to use a SeqLock when reading from / writing into the byte[] value. See Java's StampedLock API and internal algorithm notes for details
What is the GC overhead here?
Are you planning on using memory ballast for reducing GC cycles? @mangalaman93
Memory ballast doesn't make sense for ristretto and if it would make sense it's still something the application should do by itself, not the library.
A custom memory allocator doesn't make sense either seeing as ristretto only stores interface{}. You could build a library on top of ristretto that serializes all data you put in into []byte that uses a custom allocator. But that's not the use case for ristretto. ristretto allows you to put in unseralizable data like channels as well.
My intention here is to understand what is the overhead of GC that this issue tries to reduce by the memory allocator. @mangalaman93
When I see someone trying to reduce unnecessary GC cycles to keep heap memory size in check (which isn't much important) I can't help but suggest using memory ballast. Sure there needs to be fine-tuning further and it's better not to do it at the library level, but it's pretty much doable and helpful nonetheless. Also, I think it is going to be part of Go runtime soon (refer- golang/go#23044) . @erikdubbelboer Can you suggest some alternative?
I don't think this issue was created to reduce the amount of GC cycles. This is not something a custom memory allocator would change.
What this issue is about is that you want to have less pointers for the GC to check, so it runs faster. Using a custom memory allocator you can put everything in a []byte so GC only has to check 1 pointer to the []byte instead of all pointer to all objects.
I see. I would love to learn more about it. Is this what you are talking about? https://segment.com/blog/allocation-efficiency-in-high-performance-go-services/
If not, can you suggest some reading?
Both bigcache and https://github.com/coocood/freecache#how-it-is-done use different techniques to avoid having too many pointers.
@erikdubbelboer I think you know VictoriaMetric have FastCache repo as well, is it a good example for reference reading?
ethereum/go-ethereum#19971
Yes fastcache is also very good. I didn't mention it because it has a bit more complex way to deal with lowering GC overhead.
@mangalaman93 is this issue still relevant?
I think so, because the way Ristretto stores data is all pointers. The more the data, GC effects are gonna be more and more visible.
