aws-samples / amazon-ivs-auto-captions-web-demo

Use Amazon IVS in conjunction with Amazon Transcribe to deliver real-time captions for live streams.

Home Page:https://ivs.rocks/examples

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Amazon IVS Auto-captions Web demo

A demo web application for demonstrating how you can use Amazon IVS in conjunction with Amazon Transcribe to deliver real-time captions for live streams. This demo also shows how Amazon Translate can be used to deliver auto-translated captions to viewers (optional during deployment).

Auto-captions demo

This project is intended for education purposes only and not for production usage.

This is a serverless web application, leveraging Amazon IVS, Amazon Transcribe, Amazon ECS, Amazon API Gateway, AWS Lambda, Amazon DynamoDB, Amazon S3 and Amazon CloudFront. The web user interface is a single page application built using React.js and the Amazon IVS Player. The demo showcases how you can add real-time live captioning to an Amazon IVS stream using Amazon Transcribe. It also showcases how to configure image overlays to appear on top of the video player based on specific keywords, using TimedMetadata. This demo uses Amazon API Gateway WebSockets to deliver the captions to the connected clients, which are then used as a WebVTT track.


Getting Started

⚠️ Please note that this demo is experimental, and should only be used for educational purposes.

⚠️ Deploying this demo application in your AWS account will create and consume AWS resources, which will cost money.

To get the demo running in your own AWS account, follow these instructions.

  1. If you do not have an AWS account, please see How do I create and activate a new Amazon Web Services account?
  2. Log into the AWS console if you are not already. Note: If you are logged in as an IAM user, ensure your account has permissions to create and manage the necessary resources and components for this application.
  3. Follow the instructions for deploying to AWS.

Deploying to AWS


Architecture

architecture


⚠️ Known issues and limitations

  • The solution was built for demonstration purposes only and not for production use.
  • The solution requires streaming to an ECS container instead of directly to Amazon IVS, which may add points of failure and additional latency.
  • The solution is currently limited to a maximum connected viewers of ~200 (this limitation comes from the captions delivery mechanism, not Amazon IVS). Starting from 200 connected users and up, the execution time of the process to deliver the captions to connected clients increases and causes a timeout in the Lambda function (which is set at 3 seconds), resulting in captions not being delivered at all. A possible alternative approach to overcome this limitation would require replacing the WebSocket infrastructure (built on top of API Gateway, Lambda and DynamoDB) with a custom WebSocket Server implementation running in Amazon ECS and AWS Fargate. Read more here.
  • The solution's client-side caption syncing mechanism currently relies on an undocumented Player API. This API may be changed or deprecated in the future without notice.
  • In Firefox, captions may appear very close to the bottom border of the video when there are 4 or more rows of captions.
  • The solution was only tested in us-west-2 (Oregon) and us-east-1 (N. Virginia) regions. Additional regions may be supported depending on service availability.
  • You may explore using this demo as an alternative, which has fewer limitations.

Estimated costs

Deploying this solution in your AWS account will create and consume AWS resources, which will cost money.

Below is a table with estimated costs for scenarios with 1, 10, and 100 viewers, each receiving video in 1080p resolution during 1 hour with four translations enabled.

Note: These costs are estimates. Cost may vary depending on multiple factors such as (but not limited to) region, amount of viewers, duration, number of captions in the video, enabling the Translate feature, the number of translations that are activated, etc. Note that the estimated prices are in dollars and do not include taxes.


Service 1 viewer 10 viewers 100 viewers
Amazon Translate 30.78 30.78 30.78
Elastic Container Service 2.27 2.27 2.27
Interactive Video Service 2.15 3.5 17
Transcribe 0.73 0.73 0.73
CloudWatch 0.09 0.09 0.09
DynamoDB 0.02 0.25 2.5
API Gateway 0.02 0.25 2.5
Elastic Container Registry 0.09 0.09 0.09
Lambda 0.00 0.02 0.25
S3 0.00 0.00 0.00
CloudFront 0.00 0.00 0.02
Total estimated cost 36.15 37.98 56.23

About Amazon IVS

About

Use Amazon IVS in conjunction with Amazon Transcribe to deliver real-time captions for live streams.

https://ivs.rocks/examples

License:MIT No Attribution


Languages

Language:JavaScript 70.3%Language:Shell 20.0%Language:CSS 5.8%Language:Dockerfile 3.0%Language:HTML 0.8%