samkennerly / ec2box

Example Terraform module for using Amazon EC2. Each instance includes its own keypair, IAM role, security group, CloudWatch log group, and cloud-init template.

Home Page:https://samkennerly.github.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ec2box

There is no cloud. It's just someone else's computer.

Will Ferrell, wearing a 3-piece suit, sits in front of a wall of computer monitors, each of which shows Neo from the film THE MATRIX.

abstract

Use Terraform to automatically launch and configure Amazon EC2 resources. Each box includes its own:

The test module launches example free-tier Ubuntu boxes:

box name script language what it does
dorothy ruby print timestamped messages
leeroy bash print a message, then crash

basics

authorize AWS

Terraform needs access to AWS security credentials. These can be hard-coded in .tf files, but storing secrets in code can be dangerous. It is safer to use environment variables:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

or credentials files in the user's home folder:

~/.aws/config
~/.aws/credentials

choose a keypair

  • Run bin/keygen to generate a keypair.

The private and public keys will be saved here:

etc/ec2box_rsa
etc/ec2box_rsa.pub

When a new box is created, or the public key changes, Terraform will upload a copy of the public key to AWS. Anyone with both keys (public and private) can then login remotely to each box with SSH.

Caution: The private key in the ec2box_rsa file must be kept secret. This repo gitignores it.

launch boxes

Terraform will save state files when the test module is initialized:

terraform.tfstate
terraform.tfstate.backup

Caution: State files (including remote state) can contain secrets! This repo gitignores them.

inspect boxes

  • Run bin/login to login to a box remotely with SSH.

Usually, remote login is unnecessary because boxes can be monitored by reading log messages. Each box creates its own CloudWatch log group and streams logs to it. The pipeline works like this:

  1. cloud-init downloads, installs, configures, and starts an AWS CloudWatch agent.
  2. The agent creates a log stream and begins streaming from /var/log/syslog.
  3. The launch script prints errors to STDERR and all other messages to STDOUT.
  4. The shell redirects STDERR and STDOUT to the Ubuntu logger.
  5. The logger saves logs to /var/log/syslog.

Cloud-init, system, and launch script logs will then be visible in the AWS CloudWatch console.

deactivate boxes

  1. Run bin/down test to destroy all example resources.

contents

Configuration files for the Terraform root module:

Short scripts to run Terraform commands:

  • bin/clean [FOLDER] autoformats and validates Terraform code.
  • bin/down [FOLDER] destroys all resources declared in a folder.
  • bin/keygen generates and saves an RSA keypair to the etc folder.
  • bin/login [BOXNAME] uses SSH to login to an EC2 instance remotely.
  • bin/up [FOLDER] creates or updates all resources declared in a folder.

Default configuration files for each newly-created box:

  • ec2box_rsa is an RSA private key.
  • ec2box_rsa.pub is an RSA public key.
  • install is a script which installs software.
  • launch runs in the background when a box is ready to use.
  • policy.json is an IAM policy which grants AWS permissions to a box.
  • template is a template file for a cloud-init script.

Example resources for testing ec2box:

  • main.tf declares boxes to be created by terraform apply test.
  • outputs.tf declares outputs to be shown by terraform output.
  • variables.tf declares inputs to be read from terraform.tfvars.

Configuration files for a test box named dorothy which prints timestamped messages every 1 second.

dependencies

  1. AWS credentials
  2. Terraform >= 0.12
  3. OpenSSH to run bin/keygen and bin/login
  4. jq to run bin/login

examples

Create or update all test boxes. (Terraform will prompt for confirmation.)

> bin/up
Initialize and update test resources
Initializing modules...

...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

...

Remote login to dorothy via SSH. (You might need to confirm the host public key.)

> bin/login dorothy
SSH into dorothy at ubuntu@ec2-54-81-15-89.compute-1.amazonaws.com
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1051-aws x86_64)

...

ubuntu@ip-123-45-67-89:~$

Use SSH to run the launch script on the leeroy box again:

> bin/login leeroy '~/launch'
SSH into leeroy at ubuntu@ec2-18-208-220-170.compute-1.amazonaws.com
LEEROOOOOOOOOOOOOOOOOOOOOOOY JENKINS

Destroy all test boxes. (Terraform will prompt for confirmation.)

Destroy all Terraform-managed resources in test

...

Destroy complete! Resources: 14 destroyed.

faq

How do I define my own boxes?

Edit the test folder. Rename it if you want to.

Do I need to use remote state?

No, but it's usually safer than keeping local state files on one person's laptop.

How do I choose different install and/or launch scripts?

See the dorothy box in test/main.tf for an example.

How do I deploy code to a box?

Here are some common methods:

To deploy code automatically when a box is created, edit its install script.

Where are the official docs?

About

Example Terraform module for using Amazon EC2. Each instance includes its own keypair, IAM role, security group, CloudWatch log group, and cloud-init template.

https://samkennerly.github.io/

License:Other


Languages

Language:HCL 68.7%Language:Shell 28.8%Language:Ruby 2.5%