OElesin / amazon-sagemaker-nlu-search

This repository is part of a blog post that guides users through creating a NLU search application using Amazon SageMaker and Amazon Elasticsearch service

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Amazon SageMaker NLU search

This repository guides users through creating a NLU based product search using Amazon SageMaker and Amazon Elasticsearch service

How does it work?

we have used pre-trained BERT model(distilbert-base-nli-stsb-mean-tokens) from sentence-transformers to generate fixed 768 length sentence embedding on Multi-modal Corpus of Fashion Images from feidegger, a zalandoresearch dataset. Then those feature vectors is imported in Amazon ES KNN Index as a reference.

diagram

When we present a new query text/sentence, it's computing the related embedding from Amazon SageMaker hosted BERT model and query Amazon ES KNN index to find similar text/sentence and corresponds to the actual product image which is stored in Amazon S3

diagram

License

This library is licensed under the MIT-0 License. See the LICENSE file.

About

This repository is part of a blog post that guides users through creating a NLU search application using Amazon SageMaker and Amazon Elasticsearch service

License:MIT No Attribution


Languages

Language:Jupyter Notebook 53.0%Language:JavaScript 27.8%Language:Python 13.8%Language:HTML 3.4%Language:CSS 1.7%Language:Shell 0.3%