Giters

hashbrowncipher / happycache

dumps and reloads page cache; happy caches for happy databases

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Large/sparse file support

hashbrowncipher opened this issue · comments

commented

Right now we use 1/4096 of a file's size in RAM. For a 64GB file, this is 16MB. For an 8TB file, this is 2GB of RAM.

We can bound the amount of RAM used by calling mincore on parts of the file at a time. Perhaps do no more than 64GB? This would still use up to 16MB per thread, so perhaps also a global semaphore should guard overall memory usage.