tcmalloc is trying to keep everything allocated for longer
coder-saab001 opened this issue · comments
I am working on an application which is reading continuously from network and let's say storing it somewhere. I am reading in say 1 MB chunks again and again. I want the tcmalloc memory of previous iteration to be freed before next read iteration. What I can see now is it takes about ~1 sec for tcmalloc to free the previously allocated memory. Is there any way to tweak this default behaviour of tcmalloc?
You can call MallocExtension::ReleaseMemoryToSystem to speed up the process, but there's some smoothing based on past memory usage.
If another 1MB allocation is going to likely occur, it's faster to keep it cached than to return it to the OS (a syscall) and then reuse it (page faults).
We want to minimise the memory usage. Say currently 16 chunks of 1 MB are in memory. In next iteration, next 16 chunks are about to come. We want to get that first 16 MB got release before second 16MB is fed into the memory. Is that possible somehow?
I doubt that any answer given here will differ substantially from the answer given on gperftools/gperftools#1465.