google / hashr

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Large memory consumptions with large number of input files

ZetaTwo opened this issue · comments

I suspect that the cache map starts consuming a very large amount of memory after a while. I ran it on a machine with 32gb RAM and nothing other than postgres and hashr running and it killed hashr for OOM.

Should we consider adding support for offloading the cache to the database or something like redis?

hashr-crash3.log