Out of memory listing entries on one human chromosome on a machine with 300 GB ram and 165 GB BigBed file
balwierz opened this issue · comments
I thought I could use this library for querying extremely large BigWig files, but despite providing python with 300 GB of ram it crashes anyway.
Is pyBigWig keeping all the data in memory at any point of operation?
It'll need to keep those results in memory to output them. I wonder what the utility of such a large bigBed file is, since I suspect you'd be better served by storing some sort of summarization of the underlying data.