chaliy / play-datapipeline-kinesis

Application demoes data pipline Producer > AWS Kinesis > AWS Lambda > AWS DynamoDb

AWS Kinesis Serverless Demo Application

Application demoes data pipline Producer > AWS Kinesis > AWS Lambda > AWS DynamoDb.

Scenario

Product lifecycle database. During product lifecycle some events are generated. Applicaiton will process them and build some analytics.

Technical Overview

Application uses AWS Kinesis as input queue, it uses 48 hours of retention making it effectivelly peresisten queue. Processing is done using AWS Lambda. Storage is AWS DynamoDb.

How run

• Ensure dependecies: nodejs, terraform, make
• export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
• add terraform.tfvars with region and account_id
• run make apply

To generate few items:

• run node ./producer/tasks/seeds/lifecycle.js

License

MIT