[FEAT] Support simple Hybrid Search via BM25 + RRF

[Bhavnick Minhas]

Hey Minish Lab!

This is a pretty coool project! This solves one issue that I've had for a while, with using faiss and HNSW being really cumbersome to set up and use. I really appreciate this 🤎

This might be a bit out of scope for this project, so feel free to ignore this! One thing I wish to evaluate performance for usually is hybrid search with RRF (reciprocal rank fusion) of BM25 and semantic search. It's pretty cumbersome to test with weaviate where I use this usually, and vicinity is perfect for testing!

Currently, if I wish to do hybrid search with Vicinity, I would use the BM25S (sparse) library and set up my own RRF for this. But if I could have an optional installation like vicinity[bm25] or vicinity[hybrid, faiss] with a simple method to evoke hybrid search internally, it would make this a lot more convenient for me.

Please let me know if you're open to accepting this change, I would love to provide a PR for this with your support :)

Thanks for considering!
Love the project, cheers 🥂

[Thomas van Dongen]

Hey @bhavnicksm,

Thanks for the kind words! This is a really cool suggestion, and actually something that we were also thinking off a while ago. We decided that it doesn't fit directly into Vicinity since the interface for Vicinity works based on vectors and items, while with BM25 it would change to texts (essentially, one step earlier in the pipeline). However, we've been working on the idea of a different package that essentially uses Vicinity, but one step earlier (using both Model2Vec and Vicinity), where this would fit in perfectly.

Are you on Discord by any chance? If so, I could add you to our DIscord server, and we can discuss there. If you're open to it, we'd love it if you could help contribute this feature to our new package!

[Bhavnick Minhas]

Yes! I'd love to~

My discord username is also bhavnicksm

Thanks! ☺️

[Thomas van Dongen]

Great, I just send you a request!