Bloom filter sets
mappum opened this issue · comments
A possible optimization for our set operations are to use Bloom filters - we can probabilistically detect intersections in a way where the common case of 0 intersections is much faster to detect since we only have to AND the bits rather than doing O(N)
HashSet
lookups.
See more about this in the concurrency document: https://github.com/nomic-io/orga/blob/master/docs/concurrency.md#optimized-set-operations