A new wave of widespread AI adoption is on the way with generative AI,having the potential to reinvent every aspect of customer experiences and applications. Generative AI is powered by very large machine learning models that are pre-trained on vast amounts of data, commonly referred to as foundation models (FMs). Large Language Models are a subset of Foundation Models(FMs) which are trained on trillions of words and they learn the patterns in the language, allowing them to generate human-like responses to any query we give them. Additionally, foundation models are trained on very general domain corpora, making them less effective for domain-specific tasks. There lies the importance of RAG. You can use Retrieval Augmented Generation (RAG) to retrieve data from outside a foundation model and augment your prompts by adding the relevant retrieved data in context.
Text generation using RAG with LLMs enables you to generate domain-specific text outputs by supplying specific external data as part of the context fed to LLMs. With RAG, the external data used to augment your prompts can come from multiple data sources, such as a document repositories, databases, or APIs. The first step is to convert your documents and any user queries into a compatible format to perform relevancy search. To make the formats compatible, a document collection, or knowledge library, and user-submitted queries are converted to numerical representations using embedding language models. Embedding is the process by which text is given numerical representation in a vector space. RAG model architectures compare the embeddings of user queries within the vector of the knowledge library. The original user prompt is then appended with relevant context from similar documents within the knowledge library. This augmented prompt is then sent to the foundation model. You can update knowledge libraries and their relevant embeddings asynchronously.
Amazon Opensearch Serverless offers vector engine to store embeddings for faster similarity searches. The vector engine provides a simple, scalable, and high-performing similarity search capability in Amazon OpenSearch Serverless that makes it easy for you to build generative artificial intelligence (AI) applications without having to manage the underlying vector database infrastructure.
Project Updates
- Support Meta Llama2 models on Amazon Bedrock. Support for Anthropic's latest Claude 2.1 model (200K context length).
- Improve UI
![Screenshot 2023-10-27 at 1 51 17 PM](https://private-user-images.githubusercontent.com/25897220/278581532-7568e38e-ffdf-4ad1-8a2e-9e79cf11d91e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwNDgxNDQsIm5iZiI6MTcyNDA0Nzg0NCwicGF0aCI6Ii8yNTg5NzIyMC8yNzg1ODE1MzItNzU2OGUzOGUtZmZkZi00YWQxLThhMmUtOWU3OWNmMTFkOTFlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDA2MTA0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWYwMzY3NDczMzVlMjUyYTJhNmExNjcwZTIzZDMzYWU0ZTU2NmM5ZGZmMzFmYmQxZGMxMzRlNmRkY2NkOGU2NzQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.bOSK7VxwHwXqLN-JIuQxtip-ewbClhjFizPwAVjkYe4)
- Support French/German for Anthropic Claude with Amazon Bedrock
- Support for Redaction feature
- Inbuilt Text Chunking feature with RecursiveTextSplitter from Langchain
- Anthropic Claude V1/V2/Instant support over Amazon Bedrock
- Support for Streaming ingestion with Anthropic Claude Models
- Faster Stack Deployments
- New Functionality (PII/Sentiment/Translations) added on the UI
![Screenshot 2023-10-03 at 1 37 53 PM](https://private-user-images.githubusercontent.com/25897220/272189756-d90c0624-7a4b-4091-9ece-25a29f7f869f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwNDgxNDQsIm5iZiI6MTcyNDA0Nzg0NCwicGF0aCI6Ii8yNTg5NzIyMC8yNzIxODk3NTYtZDkwYzA2MjQtN2E0Yi00MDkxLTllY2UtMjVhMjlmN2Y4NjlmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDA2MTA0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTc2OTUxNzNhODllY2FlZWUyN2Y1YjI5NzhhZjI1ODc5NzQyNTUyMGQ0NGNmNzQ3YzYwZTMwMTMxNGViNjNhMjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.mo4DEPOTzT38Deltoe9FOsBUkHHG3MjxNP3_qkcAY40)
- Llama2-7B (Existing G5.2xlarge)
- Llama2-13B (G5.12xlarge)
- Llama2-70B (G5.48xlarge)
- Falcon-7B (G5.2xlarge)
- Falcon-40B (G5.12xlarge)
- Falcon-180B (p4de.24xlarge)
Bedrock RAG Demo
bedrock_conversations.mov
github-final.mov
This solution demonstrates building a RAG (Retrieval Augmented Solution) with Amazon Opensearch Serverless Vector DB and Amazon Bedrock, Llama2 LLM, Falcon LLM
Prerequisites
- An AWS account
- For Amazon Bedrock, you should have access to Anthropic Claude models
- Amazon Bedrock supported regions
- Amazon Opensearch serverless supported regions
- Amazon Sagemaker
- GPU Instance of type ml.g5.2xlarge for endpoint usage
- Supported Llama2 regions (us-east-1 , us-east-2 , us-west 2 , eu-west-1 , and ap-southeast-1)
Create an Admin User to deploy this stack
Section1 - Create an IAM user with Administrator permissions (OPTIONAL: If you're already an Admin user/role, you may skip this step)
-
Search for the service IAM on the AWS Console and go the IAM Dashboard and click on “Users“ tab under ”Access Management” and Click on “Create User”
-
Now Click on Attach Policies directly and Choose "AdminsitratorAccess" and click "Next"
-
Now we need to create credentials for this IAM. Go to "Users" tab again and you will see your new user listed over there. Now click on the username.
-
Go to Security Credentials Tab and under "Access Keys" click on "Create Access key"
![LLMAdminSecurityCredentials2](https://private-user-images.githubusercontent.com/25897220/263316438-36bdf80f-8b0e-43a4-ad0f-a3233ce753cb.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwNDgxNDQsIm5iZiI6MTcyNDA0Nzg0NCwicGF0aCI6Ii8yNTg5NzIyMC8yNjMzMTY0MzgtMzZiZGY4MGYtOGIwZS00M2E0LWFkMGYtYTMyMzNjZTc1M2NiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDA2MTA0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk5YTA1NGQ0NGI1MzYxMWNlNzdhYjgyNTE3MDFjYTVjMGU1Y2E3OWFjNzI5ODY0Mjg2ZTg0Mzk2ZDdjZTY3ZGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.d1DMP2jvO2xicSUIjnVZfrdUzBK0yzAZJN159NMG6uo)
- In the window that appears choose the first option "Command line Interface" and click the checkbox at the bottom and click Next
8.Now the Tag is optional and you can leave this empty and click on Create Access Key
Deploy the RAG based Solution (Total deployment time 40 minutes)
Section 2 - Deploy this RAG based Solution (The below commands should be executed in the region of deployment)
- Search for AWS Cloudshell. Configure your aws cli environment with the access/secret keys of the new admin user using the below command on AWS Cloudshell.
Optional if you have assumed an Administrator role.
aws configure
![LLMAdminConfigureCloudShell](https://private-user-images.githubusercontent.com/25897220/263317018-58175b14-259d-4d7d-b3e4-bb75fb48e998.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwNDgxNDQsIm5iZiI6MTcyNDA0Nzg0NCwicGF0aCI6Ii8yNTg5NzIyMC8yNjMzMTcwMTgtNTgxNzViMTQtMjU5ZC00ZDdkLWIzZTQtYmI3NWZiNDhlOTk4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDA2MTA0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIxYjU5YjIzNzMwNTlmZDJkZGQ4OWVmM2I5MzY5NWQ4ODEyMmZmYWQ3NGNiNjEwMDYyYzhkMmJiYmI2ZDllNzYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.vZBwKid0HJJ9v1T6MbK9yT8OMww7H9QWTaS5-lUVYuw)
-
Git Clone the serverless-rag-demo repository from aws-samples
git clone https://github.com/aws-samples/serverless-rag-demo.git
-
Go to the directory where we have the downloaded files.
cd serverless-rag-demo
-
Fire the bash script that creates the RAG based solution. Pass the environment and region for deployment. environment can be dev,qa,sandbox. Look at Prerequisites to deploy to the correct reqion.
sh creator.sh
-
Select the LLM you want to deploy (sh creator.sh) . Select Option 1 for Amazon Bedrock service.
-
When selecting Amazon Bedrock (Option 1), you should specify an API Key. The key should be atleast 20 characters long.
-
Press Enter to proceed with deployment of the stack or ctrl+c to exit
-
Total deployment takes around 40 minutes. Once the deployment is complete head to API Gateway. Search for API with name rag-llm-api-{env_name}. Get the invoke URL for the API
-
Invoke the Api Gateway URL that loads an html page for testing the RAG based solution as api-gateway-url/rag
- Do not forget to append "rag" at the end of the API-GW url
eg: https://xxxxxxx.execute-api.us-east-1.amazonaws.com/dev/rag
Add in your API Key used during stack Amazon Bedrock deployment to proceed with the demo