kelindar / column

High-performance, columnar, in-memory store with bitmap indexing in Go

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

possible memory leak when taking a snapshot

ami-m opened this issue · comments

I created a poc with column, it got hit with incoming data for a bout an hour, and after that I ran a loop that created a snapshot of the data once a minute.

I monitored the memory consumption and the memory usage jumped (and was not released) every time a snapshot was taken.

Other than closing the underlying writer, is there anything else I should have released?

if not, then I suspect a memory leak. pprof points the finger at the column.grow and s2.NewWriter as possible culprits.

Will need to investigate this, do you have a test case?

Yes, I also have this problem, construct a loop to execute the snapshot, you will find that there is a fixed increase in memory, the Snapshot method creates an s2.NewWriter, does not seem to free it