Try vectors from Project Panama for LSH operations
alexklibisz opened this issue · comments
Background
Project Panama vectors had some nice results for exact vector operations in #495. I'm wondering if we can use them to optimize vector hashing operations in the various LSH models.
Deliverables
- Use PP vectors in the L2LshModel hashNoProbing method: https://github.com/alexklibisz/elastiknn/blob/main/elastiknn-models/src/main/java/com/klibisz/elastiknn/models/L2LshModel.java#L69-L83
- If there's a speed-up here, try it in the probing variant.
- If there's a speed-up in both, figure out how to make it off-by-default and merge that, similar to #496
- Add benchmarks for the other model variants, and continue optimizing, likely making this checklist longer.
Related Issues
Blocked by #525 -- there should be a consistent, easy way to benchmark the changes.
I don't see any way to vectorize (or "panamize") beyond the current implementation. We're already using vectorized operations for the the dot products in this operation:
A_j dot v is the vectorized dot product of two vectors.
The next level of vectorization would require doing A dot v, i.e., the product of a matrix and a column vector. But the panama API does not have a native matrix vector multiplication. It still requires a for-loop over the rows in the matrix.
I tried to further inline the dot product function in this commit b83941f, but it had negligible effect.
One unrelated idea comes to mind: we have L hash tables and k functions per hash table. We're currently building L * k distinct vectors and computing the dot product of each one against the query/index vector. What if we just had k distinct vectors, but arranged them in L different ways, and then re-used the computed dot products but in varying arrangements.
There are some limits to this, e.g., we can only arrange 3 vectors 6 different ways, so L = 7, k = 3 would not be a valid combination.
For the fashion-mnist benchmark, we're using L = 100, k = 4. So to get 100 distinct arrangements we are computing 400 dot products. What's the minimum number of vectors we need in order to have 100 distinct arrangements of 4 vectors? I think 5: 5 * 4 * 3 * 2 * 1 = 120. So maybe we could get roughly the same effect by just computing 5 dot products instead of 400, so 80x faster.
Intuitively it seems unlikely, but maybe there's some point in the middle, e.g., do 40 dot proudcts and sample from them? Maybe this can be introduced as a new parameter, essentially describing the amount of "resampling". Could be interesting to try.
Closing this, as, 1) I don't think there is much left to be gained on the L2LshModel, 20 I'm not currently benchmarking any of the other models. Might revisit in the future for the other models or if I learn something new about the vector APIs.