kelindar / column

High-performance, columnar, in-memory store with bitmap indexing in Go

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Multiple goroutines cannot be read at the same time

nodelrd opened this issue · comments

When i use goroutines for query data

Multiple goroutines cannot be read at the same time

for i := 0; i < 6000; i++ {
	// run a query over human mages
	go measure("indexed query", "human mages", func() {
		players.Query(func(txn *column.Txn) error {
			fmt.Printf("-> result = %v\n", txn.With("human", "mage").Count())
			return nil
		})
	}, 1)

	// run a query over human mages
	go measure("indexed query", "human female mages", func() {
		players.Query(func(txn *column.Txn) error {
			fmt.Printf("-> result = %v\n", txn.With("human", "female", "mage").Count())
			return nil
		})
	}, 1)
}

And memory not release

when i use goroutines insert
cpu only used 400% , my server have 160 cpu
1646932465525

There's an examples/bench example which runs reads/writes concurrently and measures it. The library should work (but not perfectly scale) for multiple concurrent readers and writers, especially for large collections. Unfortunately I don't have access to a 160 core machine, so won't be able to reproduce at the moment, but make sure you do not update/insert at the same time. From your screenshot I'm seeing concurrent inserts which would block the reads in this case.