Giters

alexklibisz / elastiknn

Elasticsearch plugin for nearest neighbor search. Store vectors and run similarity search using exact and approximate algorithms.

Home Page:https://alexklibisz.github.io/elastiknn

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Get Fashion Mnist 96% recall up to 200 queries/second

alexklibisz opened this issue · comments

commented

I'd like to optimize Elastiknn such that the Fashion Mnist benchmark performance exceeds 200 qps at 96% recall. Currently it's at 180 qps. So this would be about an 11% improvement. There are already several issues under the performance label with ideas towards this goal. I've already merged a few PRs. I'm just opening this issue to formalize the effort and to aggregate PRs that don't otherwise have an issue.

commented

We're now above 200 qps. Last thing left is to just submit a PR to ann-benchmarks w/ the updated versions.

commented

Status update here, after releasing 8.12.2.1.

The non-containerized benchmark is reliably over 200qps @ 96% recall, around 210 qps. Latest update here: ddf637a

The containerized benchmark (running ann-benchmarks and elastiknn in the same container) has improved from ~160qps to ~180qps.

Here are the results using 8.6.2.0:

Model Parameters Recall Queries per Second
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=0 0.378 304.111
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=0 0.445 246.319
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=3 0.635 245.977
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=3 0.716 201.608
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=0 0.767 265.545
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=0 0.846 218.673
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=3 0.921 184.178
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=3 0.960 160.437

Here are the results using 8.12.2.1:

Model Parameters Recall Queries per Second
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=0 0.378 314.650
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=0 0.446 247.659
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=3 0.634 258.834
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=3 0.716 210.380
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=0 0.767 271.442
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=0 0.846 221.127
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=3 0.921 199.353
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=3 0.960 171.614