nan-wang / executor-simple-traversal-indexer

Simple Indexer based on mmap

SimpleTraversalIndexer

SimpleTraversalIndexer is based on SimpleIndexer and add support to traversal_paths so that you can index embeddings with at different granularity.

Advanced Usages

Configure the index directory

SimpleIndexer stores the Document at the directory, which is specified by workspace field under the metas attribute.

You can find how to override metas attributes at docs.jina.ai

Configure the search behaviors

You can use match_args argument to pass arguments to the match function as below.

f =  Flow().add(
     uses=SimpleIndexer,
     uses_with={
         'match_args': {
             'metric': 'euclidean',
             'use_scipy': True,
             'limit': 10}})

• For more details about overriding with configurations, please refer to here.
• You can find more about the match function at here

At search time, you can also pass arguments to config the match function. This can be useful when users want to query with different arguments for different data requests. For instance, the following codes query with a custom limit in parameters and only retrieve the top 100 nearest neighbors.

with f:
    f.search(
        inputs=Document(text='hello'), 
        parameters={'limit': 100})

Used-by

• Crossmodal Search for ImageNet
• Semantic Wikipedia Search with Transformers
• Find Similar Audio Clips

Reference

• Indexers on Jina Hub