FAISS search return indices containing -1
zhangchuheng123 opened this issue · comments
Summary
Search methods return indices containing -1
Platform
OS: Linux
Faiss version: 1.7.2
Installed from: pip
Faiss compilation options: CPU/Python
Reproduction instructions
I have encountered the problem that FAISS return some -1 indices. But I cannot find any explanation on the meaning of -1.
quantizer = faiss.IndexFlatL2(488)
index = faiss.IndexIVFFlat(quantizer, 488, 1024)
index.train(X)
index.add(X)
index.nprobs = 4
dists, inds = index.search(xq, 50)
I have checked that index.ntotal = 1027120
. I searched for xq.shape=(1024, 488)
. It returns as follows:
inds[273]
array([ 453005, 470124, 521481, 538600, 589957, 607076, 675552,
658433, 726909, 744028, 795385, 812504, 880980, 863861,
949456, 932337, 1017932, 1000813, 59285, 42167, 127757,
110639, 196229, 179111, 264701, 247583, 453006, 435887,
521482, 504363, 333173, 316055, 589958, 572839, 658434,
641315, 384530, 367411, 863862, 846743, 726910, 709791,
795386, 778267, -1, -1, -1, -1, -1,
-1])
I have also checked the X that I used to build the index: All the elements are finite and <10.
np.linalg.norm(X[453005] - xq[273])
6.2007055
np.linalg.norm(X[778267] - xq[273])
29.668596
np.linalg.norm(X[0] - xq[273])
34.85885
np.linalg.norm(X[1] - xq[273])
33.82186
Is there anyone know the reason?
I found the explanation. Closing.
https://github.com/facebookresearch/faiss/wiki/FAQ#what-does-it-mean-when-a-search-returns--1-ids