hellofiremind / migration-toolkit

Firemind Migration Toolkit Framework

Report issue . Submit a feature

Table of Contents

1. Initial Setup
1. Dependencies
2. Pre-requisites
3. Deploy IAM Role and OIDC Identity Provider
4. GitHub workflows
5. Infrastructure Setup
2. Backend and Frontend Deployments
3. Note

TL;DR: This framework allows you to get started with a Migration landing zone on AWS using native services.

Firemind’s SaaS Migration Toolkit, allows customers to get started with a Migration landing zone on AWS using native services. It’s all built using IaC and CICD. It allows customers to deploy multiple containerised workloads on to ECS, hold state in Aurora/ElastiCache/etc and deploy a frontend on S3.

Migration Toolkit is part of Firemind's Modern Data Strategy tools

Key AWS Services

• AWS ECS Fargate Container
• AWS Identity and Access Management (IAM) Roles
• Amazon Simple Storage Service (Amazon S3) Buckets
• Amazon Virtual Private Cloud (VPC)
• AWS Web Application Firewall (WAF)
• AWS Systems Manager Parameter Store (SSM) Parameters
• AWS Key Management Service (AWS KMS)
• Amazon CloudFront
• Amazon CloudWatch
• Amazon Route 53
• Amazon Relational Database Service (RDS) and Aurora

Infrastructure Diagram

Initial Setup

Dependencies

Please complete these steps before continuing.

1. Install NodeJs with npm (Install instructions in the official repo here
2. Make sure Docker is running on your machine

Prerequisites

Ensure your CLI has correct credentials to access the AWS account you want this framework deployed to.

To use this framework, create an empty remote repo in your organisation in GitHub, clone a copy of this repo and push to your remote.

Navigate to github-oidc-federation-template-infra.yml file and add a Default value for:

• GitHubOrg: This should be the name of the organisation where your repo exists.
• FullRepoName: The name of the repo which has a copy of this infrastructure.

Add the following to your remote repository secrets:

• AWS_REGION: <e.g. eu-west-1>
• S3_TERRAFORM_STATE_REGION: <e.g. eu-west-1>
• S3_TERRAFORM_STATE_BUCKET: migration-core-<account_id>-state
• ACTION_IAM_ROLE: arn:aws:iam::<account_id>:role/GithubActionsDeployMigration

Ensure you have a Route53 Hosted Zone setup in your AWS account - the domain base and DNS Zone values for this R53 Hosted Zone should be specified in the terraform-command.yml file under the TF_VAR_DOMAIN_BASE and TF_VAR_DNS_ZONE variables.

GitHub Actions deployment branches are specified in main.yml under 'push' and 'pull_request'. Ensure you add the names of your key branches here (e.g. ['development', 'staging']). GitHub Action Workflows will not run unless a commit is made to one of the branch names you have specified.

Deploy IAM Role and OIDC Identity Provider

The first step is to deploy a GitHub Actions Role and GitHub OIDC identity provider in the account that allows you to run GitHub actions for the infrastructure.

Note: This only needs to be run once per AWS account. Details on this can be found here: https://github.com/marketplace/actions/configure-aws-credentials-action-for-github-actions

• Important Note: If an identity provider already exists for your project, then remove the GithubOidc section in the github-oidc-federation-template-infra.yml file. Always check that the identity provider exists for your project, which can be found within the AWS IAM console.

Run the following command in the terminal. Ensuring the stack name is unique for the region:

aws cloudformation deploy --template-file github-oidc-federation-template-infra.yml --stack-name app-authorisation-infra-github-authentication --region {{ eu-west-1 }} --capabilities CAPABILITY_IAM --capabilities CAPABILITY_NAMED_IAM

Github Workflows

GitHub actions is used to deploy the infrastructure. The config of this can be found in the .github/workflows

We send through a variety of different environment variables

• BUILD_STAGE - We get this from the branch names
• S3_TERRAFORM_STATE_BUCKET - Get this from GitHub secrets
• S3_TERRAFORM_STATE_REGION - Get this from GitHub secrets
• AWS_REGION - Get this from GitHub secrets
• SERVICE - Has default (migration) but can be set by user in the .github/workflows files. Namely ecr-deploy.yml, frontend-deploy.yml and terraform-command.yml.
• DOMAIN_BASE - Route53 Domain Base value
• DNS_ZONE - Route53 DNS Zone value

Infastructure Setup

For quick setup follow these instructions:

• Create an empty repo within your GitHub account.
• Checkout this repository on staging branch to you local drive and push to your remote repo.
• Assuming the GitHub actions have been set up correctly, the deployment will begin.

If you are having any issues please report a bug via the repo.

Backend and Frontend Deployments

Migration ToolKit uses Terraform Modules and GitHub Actions Matrix's to enable automatic deployment for multiple frontend and backend apps in parallel. The source code for the apps are located in the './frontend' and './backend' folders respectively.

• This repo is set up with:
  • React for 2x frontend apps
  • Wordpress for 1x backend app
  • Httpd for 1x backend app
• Important Note: The property for the app names specified in backend.tf (ecs_service) and frontend.tf (frontend_name) must be identical to the folder name (i.e. './frontend/frontend1') for the app source code. For example, see how ecs_service (line 3) in backend.tf is 'app1' to match the './backend/app1' source code folder. This ensures that the GitHub Actions frontend and backend jobs deploy under the correct naming convention.
• To set up for frontend React, update the src and public folders for your project - these are located in the frontend/appname folders. Then create a frontend deployment in the frontend.tf Terraform file. The deployment process involves running npm run build and uploading the build file into the S3 site bucket. You can view this app by locating the CloudFront site and following the specified DNS.
• To setup a backend app, navigate to backend/appname and update this file with your docker image and source code. Then create a backend deployment in the backend.tf Terraform file - ensuring the health check routes, ports and other configuration options are setup correctly. Upon deployment, enter the Route53 Alias name into your web browser. This DNS name is identifiable in 'modules/backend/route53' under the ecs_alias resource.

Note

If you deploy this infrastructure and then destroy it, you must change the name for all secrets - as it is likely that the previous secret name is scheduled for deletion, and therefore reserved for a short period of time. Currently, every secret can be identified with the 'num' keyword forming part of the name.