evanwieren / ami-builder-packer

An example of an AMI Builder using CI/CD with AWS CodePipeline, AWS CodeBuild, Hashicorp Packer and Ansible.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Purpose

This Packer AMI Builder creates a new AMI out of the latest Amazon Linux AMI, and also provides a cloudformation template that leverages AWS CodePipeline to orchestrate the entire process.

Packer AMI Builder Diagram

Source code structure

├── ansible
│   ├── playbook.yaml                       <-- Ansible playbook file
│   ├── requirements.yaml                   <-- Ansible Galaxy requirements containing additional Roles to be used (CIS, Cloudwatch Logs)
│   └── roles
│       ├── common                          <-- Upgrades all packages through ``yum``
├── buildspec.yml                           <-- CodeBuild spec 
├── cloudformation                          <-- Cloudformation to create entire pipeline
│   └── pipeline.yaml
├── packer_cis.json                         <-- Packer template for Pipeline

Cloudformation template

Cloudformation will create the following resources as part of the AMI Builder for Packer:

  • cloudformation/pipeline.yaml
    • AWS CodeCommit - Git repository
    • AWS CodeBuild - Downloads Packer and run Packer to build AMI
    • AWS CodePipeline - Orchestrates pipeline and listen for new commits in CodeCommit
    • Amazon SNS Topic - AMI Builds Notification via subscribed email
    • Amazon Cloudwatch Events Rule - Custom Event for AMI Builder that will trigger SNS upon AMI completion

HOWTO

Before you start

Launch the Cloudformation stack

Region AMI Builder Launch Template
N. Virginia (us-east-1) Launch Stack
Ireland (eu-west-1) Launch Stack
London (eu-west-2) Launch Stack

To clone the AWS CodeCommit repository (console)

  1. From the AWS Management Console, open the AWS CloudFormation console.
  2. Choose the AMI-Builder-Blogpost stack, and then choose Output.
  3. Make a note of the Git repository URL.
  4. Use git to clone the repository. For example: git clone https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/AMI-Builder_repo

To clone the AWS CodeCommit repository (CLI)

# Retrieve CodeCommit repo URL
git_repo=$(aws cloudformation describe-stacks --query 'Stacks[0].Outputs[?OutputKey==`GitRepository`].OutputValue' --output text --stack-name "AMI-Builder-Blogpost")

# Clone repository locally
git clone ${git_repo}

Next, we need to copy all files in this repository into the newly cloned Git repository:

Lastly, commit these changes to your AWS CodeCommit repo and watch the AMI being built through the AWS CodePipeline Console:

git add .
git commit -m "SHIP THIS AMI"
git push origin master

AWS CodePipeline Console - AMI Builder Pipeline

Known issues

  • Currently, Packer doesn't work with ECS IAM Roles (also used by CodeBuild)
    • That's why we build a credentials file that leverages temporary credentials in the buildspec
    • When Packer supports this feature, this will no longer be necessary
  • If Build process fails and within AWS CodeBuild Build logs you find the following line Timeout waiting for SSH., it means either
    • A) You haven't chosen a VPC Public Subnet, and therefore Packer cannot connect to the instance
    • B) There may have been a connectivity issue between Packer and EC2; retrying the build step within AWS CodePipeline should work just fine

How to do an install

First we need to create the roles. Often you can create them at the same time, but often this is not the case.

aws cloudformation create-stack --template-body file://iam_cfn.yaml --parameters file://iam_params.json --tags file://tags.json --region us-east-1 --capabilities CAPABILITY_NAMED_IAM --stack-name AMIIamRoles
aws cloudformation create-stack --template-body file://pipeline.yaml --parameters file://parameters.json --tags file://tags.json --region us-east-1 --capabilities CAPABILITY_NAMED_IAM --stack-name AMIPipeline

About

An example of an AMI Builder using CI/CD with AWS CodePipeline, AWS CodeBuild, Hashicorp Packer and Ansible.

License:Apache License 2.0