facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.

Home Page:https://faiss.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

FAISS search return indices containing -1

zhangchuheng123 opened this issue · comments

Summary

Search methods return indices containing -1

Platform

OS: Linux

Faiss version: 1.7.2

Installed from: pip

Faiss compilation options: CPU/Python

Reproduction instructions

I have encountered the problem that FAISS return some -1 indices. But I cannot find any explanation on the meaning of -1.

quantizer = faiss.IndexFlatL2(488)
index = faiss.IndexIVFFlat(quantizer, 488, 1024)
index.train(X)
index.add(X)
index.nprobs = 4

dists, inds = index.search(xq, 50)

I have checked that index.ntotal = 1027120. I searched for xq.shape=(1024, 488). It returns as follows:

inds[273]
array([ 453005,  470124,  521481,  538600,  589957,  607076,  675552,
        658433,  726909,  744028,  795385,  812504,  880980,  863861,
        949456,  932337, 1017932, 1000813,   59285,   42167,  127757,
        110639,  196229,  179111,  264701,  247583,  453006,  435887,
        521482,  504363,  333173,  316055,  589958,  572839,  658434,
        641315,  384530,  367411,  863862,  846743,  726910,  709791,
        795386,  778267,      -1,      -1,      -1,      -1,      -1,
            -1])

I have also checked the X that I used to build the index: All the elements are finite and <10.

np.linalg.norm(X[453005] - xq[273])
6.2007055

np.linalg.norm(X[778267] - xq[273])
29.668596

np.linalg.norm(X[0] - xq[273])
34.85885

np.linalg.norm(X[1] - xq[273])
33.82186

Is there anyone know the reason?