Giters

Baqend / Orestes-Bloomfilter

Library of different Bloom filters in Java with optional Redis-backing, counting and many hashing options.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

adding support or hash function : xxhash

mezuqu opened this issue · comments

commented

According to the xxHash function's website: http://cyan4973.github.io/xxHash/
xxHash is the fastest non-cryptological function. Is there a plan to implement xxHash in roadmap ?

commented

Thanks for the hint, would be interesting to add that. However, if you look at the numbers at https://github.com/Baqend/Orestes-Bloomfilter#hash-functions the hash function is hardly the bottleneck. Using Murmur3 you get a more widespread implementation that manages 100,000 * 10 hashes in 20ms resulting in ~10 mio Bloom filter additions per second on a commodity machine. I wonder in which use cases that is a bottleneck and how much xxHash could improve on that throughput.

commented

Closed for now, but we would be happy to take a pull request.

commented

in our case that amount hash request per sec is huge actually, also 20ms waiting time is really huge for a system. considering bloomfilter concept is for speed :)

I would be happy if you could find the time to implement xxhash. thanks!