Improve batcher and request pool interaction
yacovm opened this issue · comments
Currently, the Batcher has a busy-wait (with sleep) loop that waits for a batch to be filled by the request pool:
func (b *BatchBuilder) NextBatch() [][]byte {
currBatch, full := b.pool.NextRequests(b.maxMsgCount, b.maxSizeBytes)
if full {
return currBatch
}
timeout := time.After(b.batchTimeout) //TODO use task-scheduler based on logical time
for {
select {
case <-b.closeChan:
return nil
case <-timeout:
currBatch, _ = b.pool.NextRequests(b.maxMsgCount, b.maxSizeBytes)
return currBatch
default:
time.Sleep(10 * time.Millisecond)
if len(currBatch) < b.pool.Size() { // there is a possibility to extend the current batch
currBatch, full = b.pool.NextRequests(b.maxMsgCount, b.maxSizeBytes)
if full {
return currBatch
}
}
}
}
}
This is slow as it sleeps, and also wastes CPU cycles.
Ideally, we would be able to reach into the request pool with a context that can fire when a timeout expires, and some additional information about the batch size, etc. that hints the request pool when to return the requests.
Long term we can consider giving the request pool the ability to build batches of its own, and remove the Batcher altogether .