CSV Repo

App Link: https://d3i442e6zsckr8.cloudfront.net/.

Architecure Diagram

Solution

Amplify

Most things start from here. I created an Amplify application which I set up most other services from (e.g. S3). This is why some of my services have names that look like they are generated by a program. Though the only thing I use through Amplify is AWS Cognito for authentication, where I use @aws-amplify/ui-vue to display an authentication component that uses amplify frontend APIs to communicate with AWS Cognito for authentication.

Lambda Function: Set User Group Handler

A function that is triggered when a user is created in AWS Cognito to set the default group for the user, which is the full-access group.

An admin must access the Cognito Pool page to change the group for a user.

API Gateway

The API gateway is configured to call the CSV resource handler lambda function for any request.

Lambda Function: CSV Resource Handler

The application logic is written as an express app unaware of its architectural context, a lambda function, and then an abstraction layer, particularly, aws-serverless-express, is used in the lambda function handler to direct the request to the correct controller.

This design has been useful, not only because the architecture is cleaner with not much fuss caused by the many to-be lambda functions, but also because the application logic was written as a traditional app, freeing me from having to handle some tricky parts if I were to create a lambda function for each controller, like authorization.

Here is a sample code showing how authorization is handled in my app:

const express = require('express')

function authorize(allowedGroups) {
  return async (req, res, next) => {
    const [_, token] = req.headers.authorization.split('Bearer ')
    try {
      await cognitoJwtVerifier.verify(token, {
        tokenUse: null,
        clientId: null,
        groups: allowedGroups
      })
      next()
    } catch {
      res.status(401).end()
    }
  }
}

app.get('/csv', authorize(['full-access', 'read-write-access', 'read-access']), (req, res) => {
  // logic ..
})

S3

Used by the CSV resource handler lambda function through the AWS JS SDK to upload files to.

SQS/SNS/DynamoDB

SQS is Treated as a thread that the CSV resource handler sends requests to, to load the uploaded CSV files to DynamoDB as a table by publishing to an SNS topic that the queue is subscribed to.

The CSV files’ tables uploaded by my application start with “obada_csv_repo”.

Confessions:

The SQS queue name is incorrect, and I realized that at the end of the assignment and didn't have (بتع) enough to fix it :)
In my lambda functions there are place(s) where I should use environment variables to hide secret values but I didn't :)
I have a bug where if you try to upload a file with a name that already exists will cause its DynamoDB table to be created and deleted endlessly. Not sure why, but I think it has something to do with the queue storing old files names and sending them to the lambda handler.

obadakhalili / csv-repo