This project is an implementation of Twitter's Finagle to trigger AWS Lambda with REST API through AWS API Gateway.
Find AWS Lambda with gRPC Implementation here: AWS-Lambda-with-gRPC.
- Author: Lakshmanan Meiyappan
- NetID: lmeiya2@uic.edu
Steps to install the project:
- Clone the project from GitHub using
git clone
git clone https://github.com/laxmena/finagle-client.git
- Install the dependencies using
sbt
sbt clean compile assembly sbt test
- Run the project using
sbt run
sbt "run [date] [timeStamp] [window]"
Example: sbt "run 2021-11-02 1:30:45 1"
- Date Format:
YYYY-MM-DD
- Time Format:
HH:MM:SS
- Window: integer value in minutes
- Gets input from client and validates the input
- Calls the AWS Gateway API to trigger Lambda function
- Returns the response from Lambda function
Language: Scala | Framework: Finagle
Find more about Lambda Functions and check the source code here: LambdaCode
- Supports the following HTTP methods: GET, POST
- Supports the following HTTP headers:
Content-Type
:application/json
- Requires the following input parameters:
date
:YYYY-MM-DD
timeStamp
:HH:MM:SS
window
: integer value in minutes
- Checks LogFile Index to get the list of files, and starting byte position for the given timestamp within the file.
- Loads the file from S3 bucket, from the starting byte position.
- Checks for all pattern matches in the logfiles in the specified window.
- Converts the matches into
md5 hashes
and returns it to client. - If there is no logs available for specified window, it returns
404
with message.
Language: Python | Dependencies: boto3, hashlib, json, pickle
sbt "run 2021-10-20 21:46:55 1"
{"isAvailable": true, "matchHashes": ["95f52588921f1fd246f39696ca423556", "04c3d10b1f6b0a75f49429e87c08d202", "648fc8438d9727b1db25ac203b7e96d0", "aeaae59adf2039491054eef598a7ce27",
"aba582241e5ac179b31cbd205ef6b905", "b6a49fd02f1a38955ebd47b6e39d1c07", "7eda16cc510b49c46f0966c8f3d28b06", "af75ac666f6d4293f1adbdc8905575e2", "87e87e0c4a1b73c0dfbf66b71880b90d", "f7bfe076dd461afe326ebae315b61858", "c30d423269a286a
cfebea871c209a0e4", "2833ca97b7a78938511b76190f1ff7bc", "96aafa47b0eb524a3d0e63ffbdc7f738", "b3320a0a11374a5575685bfb04be1f43", "a70216a183cb2e8094b6513df4204d6c", "b6da697709c1149a687a0e4bfd737710", "916fd64ce7e5b8fbb3a86cddd2d88c3
7", "6ff7a20520d511468283a67ff9e8fea7", "04958cf093aae20123aef74cc763d854", "a54975d81e3ee84abe69898ad125fc82"]}
AWS Lambda function can also be triggered using Postman. Please find the screenshot below.
- Twitter Finagle Guide: https://twitter.github.io/finagle/
- Twitter Finagle GitHub : https://github.com/twitter/finagle
- AWS Lambda : https://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction.html
- AWS API Gateway : https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-rest-api.html
- AWS IAM : https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html
- Boto3: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda.html
- AWS S3 Documentation: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingObjects.html
- Scala Regex Documentation: https://www.scala-lang.org/api/current/scala/util/matching/Regex.html
- LogProcessing-MapReduce Project: https://github.com/laxmena/LogProcessing-MapReduce