Block Select Profiles should be correctly merged across row group
cyriltovena opened this issue · comments
For a given block, between each row group we should also do a k-way merge since the series order is not preserved when flushing to disk.
This can cause some incorrectness when deduping as we expect data to be streamed in order.
I think this requires a big refactoring on parquet iterators, we need a way to produce one iterator per rowgroup. Then use this merge https://github.com/grafana/phlare/blob/63d522a723082453d54b1ece0a462f86887fac66/pkg/iter/profiles.go#LL38C27-L38C27
Actually just looked into this again and we actually don't have a bug, we load all profiles in memory for a given block when selecting profiles and merge them.
see https://github.com/grafana/phlare/blob/main/pkg/phlaredb/block_querier.go#L948-L975
So in the end no bug.