Bedrock Claude Chat

This repository is a sample chatbot using the Anthropic company's LLM Claude 3 Sonnet, Claude 2, and Claude Instant, 3 of the foundational models provided by Amazon Bedrock for generative AI.

Deploy using CDK

This section describes the procedure for deploying the chat bot using CDK (the Cloud Development Kit).

This video explains the deployment step by step:

Enable access to Claude model on Bedrock

Open Bedrock Model access > Edit > Check Claude and Save changes

Please have UNIX and a Node.js runtime environment. If not, you can also use Cloud9
Clone this repository

git clone https://github.com/cabcookie/bedrock-claude-chat.git

Install npm packages

cd bedrock-claude-chat
cd cdk
npm ci

Install AWS CDK

npm i -g aws-cdk

Configure access to AWS Account

You need to be signed in to the appropriate AWS account before you run the deployment. If you are not sure how to sign in to your AWS account from the command line, read Configure the AWS CLI.

Before deploying the CDK, you will need to work with Bootstrap once for the region you are deploying to. In this example, we will deploy to the us-east-1 region. Please replace your account id into <account id>.

cdk bootstrap aws://<account id>/us-east-1

Edit the following entries in cdk.json according to your preferences.
- bedrockRegion: Region where Bedrock is available. NOTE: Bedrock does NOT support all regions for now.
- domainAlias: Provide a domain name like chat.myorganization.com. Please be aware that you need to adjust certain DNS settings with your domain provider (see more details below). Please refer to Setting up a domain alias for more details
- hostedByRoute53: If you have set up a hosted zone in Route 53 for your domain (like myorganization.com) you can let Route 53 manage the domain for the web frontend as well as for the API endpoints,
- allowedIpV4AddressRanges, allowedIpV6AddressRanges: Allowed IP Address range.
Deploy this sample project

cdk deploy --all

The script may stop at the stage AWS::CertificateManager::Certificate if you configured a domainAlias in the cdk.json. Please refer to Setting up a domain alias for more details.

You will get output similar to the following. The URL of the web app will be output in BedrockChatStack.FrontendURL, so please access it from your browser.

 ✅  BedrockChatStack

✨  Deployment time: 78.57s

Outputs:
BedrockChatStack.AuthUserPoolClientIdXXXXX = xxxxxxx
BedrockChatStack.AuthUserPoolIdXXXXXX = us-east-1_XXXX
BedrockChatStack.BackendApiBackendApiUrlXXXXX = https://xxxxx.execute-api.us-east-1.amazonaws.com
BedrockChatStack.DomainAliasURL = https://chat.myorganization.com
BedrockChatStack.FrontendURL = https://xxxxx.cloudfront.net

📚 Supported Languages

English 💬
日本語 💬
한국어 💬
中文 💬

Use the chat bot

Visit the page https://chat.myorganization.com (mentioned in BedrockChatStack.DomainAliasURL) or https://xxxxx.cloudfront.net (mentioned in BedrockChatStack.FrontendURL).

The sign-up screen will appear as shown above, where you can register your email and log in.

Important This deployment method allows anyone with the URL to sign up. For production use, we strongly recommend adding IP address restrictions or disabling self-signup to mitigate security risks. Read more on how to Disable self sign up.

Setting up a domain alias

If you configured a domainAlias in the cdk.json the deployment will stop at the stage AWS::CertificateManager::Certificate. You need to visit your domain service provider and configure a DNS entry to confirm you own the domain. Please visit AWS Certificate Manager. You will find the domain alias in the list in the status Pending validation. It indicates that the Certificate Manager is validating domain ownership.

If I click on the Certificate ID I can see the CNAME name and the CNAME value the Certificate Manger is looking for:

I need to set those values in my DNS settings at my domain provider:

It takes a couple of minutes until the status changes to Issued.

Please find more details in this documentation.

Architecture

It's an architecture built on AWS managed services, eliminating the need for infrastructure management. Utilizing Amazon Bedrock, there's no need to communicate with APIs outside of AWS. This enables deploying scalable, reliable, and secure applications.

Amazon DynamoDB: NoSQL database for conversation history storage
Amazon API Gateway + AWS Lambda: Backend API endpoint (AWS Lambda Web Adapter, FastAPI)
Amazon SNS: Used to decouple streaming calls between API Gateway and Bedrock because streaming responses can take over 30 seconds in total, exceeding the limitations of HTTP integration (See quota).
Amazon CloudFront + S3: Frontend application delivery (React, Tailwind CSS)
AWS WAF: IP address restriction
Amazon Cognito: User authentication
Amazon Bedrock: Managed service to utilize foundational models via APIs

Features and Roadmap

Others

Configure text generation parameters

Edit config.ts and run cdk deploy.

export const GENERAL_CONFIG = {
  max_tokens_to_sample: 500,
  temperature: 0.6,
  top_k: 250,
  top_p: 0.999,
  stop_sequences: ["Human: ", "Assistant: "],
};

Remove resources

If using cli and CDK, please cdk destroy. If not, access to CloudFormation then delete BedrockChatStack and FrontendWafStack manually. Please note that FrontendWafStack is on us-east-1 region.

Language Settings

This asset automatically detects the language using i18next-browser-languageDetector.You can switch languages from the application menu. Alternatively, you can use Query String to set the language as shown below.

https://example.com?lng=ja

Disable self sign up

This sample has self sign up enabled by default. To disable self sign up, open auth.ts and switch selfSignUpEnabled as false, then re-deploy.

const userPool = new UserPool(this, "UserPool", {
  passwordPolicy: {
    requireUppercase: true,
    requireSymbols: true,
    requireDigits: true,
    minLength: 8,
  },
  // true -> false
  selfSignUpEnabled: false,
  signInAliases: {
    username: false,
    email: true,
  },
});

Local Development

See LOCAL DEVELOPMENT.

Contribution

Thank you for considering contribution on this repository! We welcome for bug fixes, language translation, feature enhancements, and other improvements. Please see following:

RAG using Kendra

In this sample, we have not implemented RAG using Kendra. This is because when it comes to real-world deployments, factors such as access control policies, the presence or absence of data connectors, and the methods for authentication and authorization for the connected data sources can be quite diverse depending on the organization, making it difficult to generalize them in a simple manner. To put this into practice, you should consider downsides like decreased latency and increased token consumption. For these reasons, a proof of concept (PoC) to verify search accuracy is essential.

generative-ai-use-cases-jp (In Japanese)
simple-lex-kendra-jp (In Japanese)
jp-rag-sample (In Japanese)

Authors

License

This library is licensed under the MIT-0 License. See the LICENSE file.

cabcookie / bedrock-claude-chat