Avoid calculating the sum of items in bitmap statistics
Dr-Emann opened this issue · comments
Zachary Dremann commented
Calculating the sum of values requires visiting every item in the bitmap: Other than this, all other statistics can be calculated by only visiting each container. Is it worth either:
- Removing the field entirely (breaking change, both API and ABI)
- Changing the implementation to leave sum as zero, and documenting that the sum field is not actually updated.
Originally posted by @AviAvni in #617 (comment)
Avi Avni commented
we can also provide another api to get a sum just like min and max and it can be optimize for example if we can iterate on ranges for example 1..100 1000..2000 we can just simply calculate (1+100)*100/2+(1000+2000)*1000/2
Daniel Lemire commented
I think it would be an acceptable breaking change.