rogeriomm / aws-lab

Terraform AWS free tier, EC2/ECR/RDS/EFS/DynamoDB/Lambda/S3. Docker running on EC2, Traefik reverse proxy, Lets Encrypt, dynamic DNS, OpenVPN(TODO)

Terraform AWS Free Tier

Getting started with the Terraform for managing a base free-tier AWS resources.

Project description

This is a Terraform project for managing AWS resources.

It can build the next infrastructure:

• VPC
• Public Subnet in the VPC
• IGW to enable access to or from the Internet for VPC
• Route Table to associate IGW, VPC and Subnet
• RDS in the private Subnet
• EC2 in the public Subnet with the HTTPS Lets Encrypt certificate & SSH access with free dynamic DNS provider
  • Docker running on EC2: docker-compose.yaml
    • Free dynamic DNS host on AWS
    • Web application sample
      • My personal URL: https://aws-dashboard.duckdns.org/whoami
    • Traefik Reverse proxy, Lets Encrypt, Traefik Lets Encrypt
      • My personal URL: https://aws-dashboard.duckdns.org/traefik
      • 
• S3
• Lambda
  • Samples
    • Python Source code
    • Golang Source code
    • Java Source code
    • RUST, RUST Runtime for AWS Lambda: Source code
• EFS
  • Mount NFS filesystem on AWS EC2
• DynamoDB
• ECR
• GLUE

Architecture Diagram

Install

Pre steps

1. Install software
   • Install infrastructure tools

brew install terraform awscli

• Install general development tools

brew install make yq zip curl gradlle

• Install PYTHON

brew install pyenv

• Install GO

brew install golang

• Install JAVA

brew install java11 jenv

• Install RUST: https://rustup.rs

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup target add x86_64-unknown-linux-gnu
brew tap cargo-lambda/cargo-lambda
brew install cargo-lambda

2. Create AWS account
3. If the file ~/.aws/credentials doesn't exist, create it and add you Terraform profile to the file. For example:

   [terraform]
   aws_access_key_id = Your access key
   aws_secret_access_key = Your secret access key 

4. Check AWS account

aws sts get-caller-identity

5. Create S3 bucket to store Terraform state

aws s3api create-bucket --bucket world-terraform --region us-east-1

6. Create config file config.tf that will contain information how to store state in a given bucket. See example.

7. Create SSH key pair to connect to EC2 instance:

cd ./src/free-tier/provision/access

# it creates "free-tier-ec2-key" private key and "free-tier-ec2-key.pub" public key
ssh-keygen -f free-tier-ec2-key

9. Adds SSH private key identities to the authentication agent

ssh-add src/free-tier/provision/access/free-tier-ec2-key

Build infrastructure

• Install Python

pyenv install 3.8.16
cd ./src/free-tier/python
pyenv local 3.8.16

• Install Java

jenv enable-plugin gradle

• Install RUST lambda

brew tap cargo-lambda/cargo-lambda
brew install cargo-lambda

• Build lambda samples

cd ./src/free-tier/lambda/samples
make all

cd ./src/free-tier
terraform init -backend-config="./backend/config.tf"

cd ./src/free-tier
terraform plan

cd ./src/free-tier
terraform apply

Post install

ip=$(aws ec2 describe-instances | 
      yq 'select(.Reservations[].Instances[].State.Code == 16) | .Reservations[].Instances[].NetworkInterfaces[].PrivateIpAddresses[].Association.PublicIp')
echo $ip

• Edit /etc/hosts add "aws" host name

sudo bash -c "echo $ip aws >> /etc/hosts"

• Add SSH public key from EC2 server

ssh-keygen -R aws
ssh-keyscan -H aws >> ~/.ssh/known_hosts

• Install EPEL, Postgres 14 packages

ssh ec2-user@aws "sudo amazon-linux-extras install epel postgresql14 -y"

• Upgrade Linux and install packages

ssh ec2-user@aws "sudo yum update && sudo yum upgrade -y && sudo yum install -y netcat openvpn postgresql docker python3-pip htop"

• Install Docker compose, configuration

ssh ec2-user@aws "sudo usermod -a -G docker ec2-user && sudo pip3 install docker-compose"
ssh ec2-user@aws "sudo systemctl enable docker.service && sudo systemctl start docker.service && systemctl status docker.service"

• Edit src/docker/env-duckdns.sh

SUBDOMAINS=your-subdomain
DUCKDNS_TOKEN=your-token
TOKEN=$DUCKDNS_TOKEN

• Edit src/docker/docker-compose.yaml, set email and Duckdns subdomain TODO

• Edit src/docker/conf/users.txt TODO

• Copy Docker files

ssh ec2-user@aws "mkdir -p docker/conf"
scp src/docker/docker-compose.yaml src/docker/env-duckdns.sh ec2-user@aws:./docker/
scp src/docker/conf/dynamic_conf.yml ec2-user@aws:./docker/conf/dynamic_conf.yml
scp src/docker/conf/users.txt ec2-user@aws:./docker/conf/users.txt

• Start Docker containers

ssh ec2-user@aws "cd docker && docker-compose up -d"

Services

Duckdns

• https://www.duckdns.org/

• ./src/docker/env-duckdns.sh

SUBDOMAINS=sub-domain-1,sub-domain-2,sub-domain-3,sub-domain-4,sub-domain-5
DUCKDNS_TOKEN=your-token
TOKEN=your-token

• Duckdns logs

ssh ec2-user@aws "cd docker && docker-compose logs duckdns"

Traefik

• Traefik logs

ssh ec2-user@aws "cd docker && docker-compose logs traefik"

AWS RDS

aws rds describe-db-instances | yq

• Get Postgres endpoint

address=$(aws rds describe-db-instances | yq '.DBInstances[] | select(.DBName=="labdb") | .Endpoint.Address')
port=$(aws rds describe-db-instances | yq '.DBInstances[] | select(.DBName=="labdb") | .Endpoint.Port')
echo $address:$port

• Check Postgres routing from EC2

ssh ec2-user@aws "nc -v $address $port"

• Postgres cli

ssh ec2-user@aws psql --host $address --port $port --username postgres

AWS DynamoDB

aws dynamodb list-tables | yq

AWS EFS

aws efs describe-file-systems | yq

aws efs describe-access-points | yq

filesystem_id=$(aws efs describe-file-systems | yq '.FileSystems[] | select(.Name=="free-tier-efs") | .FileSystemId')
echo $filesystem_id

aws efs describe-mount-targets --file-system-id $filesystem_id | yq

• Get NFS IP address

nfs_ip=$(aws efs describe-mount-targets --file-system-id $filesystem_id | yq '.MountTargets[] | select (.AvailabilityZoneName=="us-east-1a") | .IpAddress')
echo $nfs_ip

• Check NFS routing from EC2

ssh ec2-user@aws "nc -v $nfs_ip 2049"

AWS API Gateway

aws apigatewayv2 get-apis | yq

endpoint=$(aws apigatewayv2 get-apis | yq '.Items[] | select (.Name=="free-tier-api-gateway") | .ApiEndpoint')
echo $endpoint

• Run lambda Python

curl "$endpoint/api/python"

Open VPN

• MAC OS client: https://tunnelblick.net/downloads.html

Destroy infrastructure

cd ./src/free-tier
terraform destroy

Links to similar projects

• https://github.com/pvarentsov/terraform-aws-free-tier
• https://github.com/deersheep330/terraform-aws-ecs-free-tier
• https://github.com/HelloMinchan/aws-free-tier-infrastructure
• https://github.com/gruberdev/tf-free

Links

• https://github.com/awslabs/aws-lambda-rust-runtime
• https://github.com/awsdocs/aws-lambda-developer-guide