Memory Error GPU 100M vectors

[Arina Razmyslovich]

Summary

Platform

OS: Amazon Linux 2

Faiss version: faiss-gpu 1.7.2

Installed from: pip install faiss-gpu

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

I'm trying to run a similarity search for text embeddings. It contains around 100M embeddings, but there's an error after adding the remained data to the index.
Also, I'm confused whether I need to train the data for similarity search using a composite index. It wasn't clear for me from the examples in the repo, could you please help with this?

import faiss

index = faiss.index_factory(384, "PCA64,IVF1048576_HNSW32,Flat")

gpu_index = faiss.index_cpu_to_all_gpus(index) # build the index

for emb in all_sent_embs:
    faiss.normalize_L2(x = emb)
    gpu_index.add(emb)           # add vectors to the index

RuntimeError: Exception thrown from index 0: Error in virtual void* faiss::gpu::StandardGpuResourcesImpl::allocMemory(const faiss::gpu::AllocRequest&) at /project/faiss/faiss/gpu/StandardGpuResources.cpp:452: Error: 'err == cudaSuccess' failed: StandardGpuResources: alloc fail type IVFLists dev 0 space Device stream 0x558aefe62230 size 16384 bytes (cudaMalloc error out of memory [2])