Remove bulk benchmark
jcarreira opened this issue · comments
Add benchmark for 'remove bulk' command. @devloop0 says this operation is quite slow.
Test with:
Number of keys in object store: 5000
Number of keys to remove: 5000
Size of each key: 100KB
'Remove bulk' is not batched. It takes 400-500ms to remove 5000 objects.
Need to check if remove bulk actually releases memory.
Test:
Add 20GB of objects. Check memory usage. Remove all objects. Check memory usage.
Confirmed this problem.
It seems this is caused by libc not freeing memory. If I run this case with the server on valgrind I observe two things:
- Memory actually gets freed (it goes down on htop)
- If I abort the process right after removing 20GB of 1MB objects I get the following valgrind output:
22155 ==3360== LEAK SUMMARY:
22156 ==3360== definitely lost: 0 bytes in 0 blocks
22157 ==3360== indirectly lost: 0 bytes in 0 blocks
22158 ==3360== possibly lost: 62,080 bytes in 170 blocks
22159 ==3360== still reachable: 1,511,349 bytes in 20,313 blocks
22160 ==3360== suppressed: 0 bytes in 0 blocks
This seems to be a problem with libc not freeing memory to the OS. Closing for now.