Integrate with ann-benchmarks as primary benchmarking tool

[Alex Klibisz]

Background

The current custom benchmark suite is difficult to use, maintain, and share results. A custom solution made sense originally, as it was intended for large-scale distributed benchmarks, e.g., in Kubernetes. However, it turns out that local, single-node benchmarks are really all we need to tell if performance is improving or degrading, and should be simpler to maintain.

Ann-benchmarks is the defacto standard benchmark. It provides useful feedback and is reasonably simple to run on a single node, e.g., on a developer laptop. So, we'll switch to using ann-benchmarks as the primary benchmarking tool for Elastiknn.

Deliverables

• Add ann-benchmarks as a git submodule.
• Taskfile tasks to setup the submodule and environment.
• Taskfile tasks to run containerized fashion MNIST benchmark.
• Taskfile tasks to execute MINIST benchmark against an elasticsearch instance on localhost:9200.
• Taskfile tasks to generate markdown file with latest benchmark figures and tables.
• Tests in CI to validate the benchmarks are working on a small dataset.
• Remove the existing Scala benchmarks.
• Remove the existing ann-benchmarks example.

Related Issues

Several previous benchmarking iterations:

• #58
• #33
• #278