Giters

kelindar / column

High-performance, columnar, in-memory store with bitmap indexing in Go

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The performance seems low...

didip opened this issue · comments

commented

I modified the million benchmark to billion and it took forever to insert.

Is it possible to expand the commit.Buffer to speed up the transaction?

commented

Hello, I also noticed a very high bump in insert-time-per-record when bumping amount to 1B from 10M (as in, more than 100x time for 100x records). While the mass-loading pattern doesn't exactly complement a store built for concurrent DML, I agree the 1B benchmark should insert in a reasonable time regardless - especially with all data being known before hand.

commented

Indeed, insertion is a bit slow as it touches most columns. I have a few questions to clarify this.

  1. How did you perform the insertion (single or multiple txn)?
  2. What was the time to insert 1B?
  3. What's the expected time?