nodomain / terraform-aws-jenkins

Terraform module to build Docker image with Jenkins, save it to an ECR repo, and deploy to Elastic Beanstalk running Docker stack

terraform-aws-jenkins

terraform-aws-jenkins is a Terraform module to build a Docker image with Jenkins, save it to an ECR repo, and deploy to Elastic Beanstalk running Docker.

This is an enterprise-ready, scalable and highly-available architecture and the CI/CD pattern to build and deploy Jenkins.

Features

The module will create the following AWS resources:

• Elastic Beanstalk Application
• Elastic Beanstalk Environment with Docker stack to run the Jenkins master
• ECR repository to store the Jenkins Docker image
• EFS filesystem to store Jenkins config and jobs (it will be mounted to a directory on the EC2 host, and then to the Docker container)
• CodePipeline with CodeBuild to build and deploy Jenkins so even Jenkins itself follows the CI/CD pattern
• CloudFormation stack to run a DataPipeline to automatically backup the EFS to S3
• CloudFormation stack for SNS notifications about the status of each backup

After all of the AWS resources are created,

CodePipeline will:

• Get the specified Jenkins repo from GitHub, e.g. https://github.com/cloudposse/jenkins
• Build a Docker image from it
• Save the Docker image to the ECR repo
• Deploy the Docker image from the ECR repo to Elastic Beanstalk running Docker stack
• Monitor the GitHub repo for changes and re-run the steps above if new commits are pushed

DataPipeline will run on the specified schedule and will backup all Jenkins files to an S3 bucket by doing the following:

• Spawn an EC2 instance
• Mount the EFS filesystem to a directory on the EC2 instance
• Backup the directory to an S3 bucket
• Notify about the status of the backup (Success or Failure) via email
• Destroy the EC2 instance

---

This project is part of our comprehensive "SweetOps" approach towards DevOps.

It's 100% Open Source and licensed under the APACHE2.

Usage

For complete examples, see examples.

Examples

Deploy Jenkins into an existing VPC with existing subnets

variable "max_availability_zones" {
  default = "2"
}

data "aws_availability_zones" "available" {}

module "jenkins" {
  source      = "git::https://github.com/cloudposse/terraform-aws-jenkins.git?ref=master"
  namespace   = "cp"
  name        = "jenkins"
  stage       = "prod"
  description = "Jenkins server as Docker container running on Elastic Beanstalk"

  master_instance_type         = "t2.medium"
  aws_account_id               = "000111222333"
  aws_region                   = "us-west-2"
  availability_zones           = ["${slice(data.aws_availability_zones.available.names, 0, var.max_availability_zones)}"]
  vpc_id                       = "vpc-a22222ee"
  zone_id                      = "ZXXXXXXXXXXX"
  public_subnets               = ["subnet-e63f82cb", "subnet-e66f44ab", "subnet-e88f42bd"]
  private_subnets              = ["subnet-e99d23eb", "subnet-e77e12bb", "subnet-e58a52bc"]
  loadbalancer_certificate_arn = "XXXXXXXXXXXXXXXXX"
  ssh_key_pair                 = "ssh-key-jenkins"

  github_oauth_token  = ""
  github_organization = "cloudposse"
  github_repo_name    = "jenkins"
  github_branch       = "master"

  datapipeline_config = {
    instance_type = "t2.medium"
    email         = "[e-mail]"
    period        = "12 hours"
    timeout       = "60 Minutes"
  }

  env_vars = {
    JENKINS_USER          = "admin"
    JENKINS_PASS          = "123456"
    JENKINS_NUM_EXECUTORS = 4
  }

  tags = {
    BusinessUnit = "ABC"
    Department   = "XYZ"
  }
}

Deploy Jenkins into an existing VPC and new subnets

variable "max_availability_zones" {
  default = "2"
}

data "aws_availability_zones" "available" {}

module "jenkins" {
  source      = "git::https://github.com/cloudposse/terraform-aws-jenkins.git?ref=master"
  namespace   = "cp"
  name        = "jenkins"
  stage       = "prod"
  description = "Jenkins server as Docker container running on Elastic Beanstalk"

  master_instance_type         = "t2.medium"
  aws_account_id               = "000111222333"
  aws_region                   = "us-west-2"
  availability_zones           = ["${slice(data.aws_availability_zones.available.names, 0, var.max_availability_zones)}"]
  vpc_id                       = "vpc-a22222ee"
  zone_id                      = "ZXXXXXXXXXXX"
  public_subnets               = "${module.subnets.public_subnet_ids}"
  private_subnets              = "${module.subnets.private_subnet_ids}"
  loadbalancer_certificate_arn = "XXXXXXXXXXXXXXXXX"
  ssh_key_pair                 = "ssh-key-jenkins"

  github_oauth_token  = ""
  github_organization = "cloudposse"
  github_repo_name    = "jenkins"
  github_branch       = "master"

  datapipeline_config = {
    instance_type = "t2.medium"
    email         = "[e-mail]"
    period        = "12 hours"
    timeout       = "60 Minutes"
  }

  env_vars = {
    JENKINS_USER          = "admin"
    JENKINS_PASS          = "123456"
    JENKINS_NUM_EXECUTORS = 4
  }

  tags = {
    BusinessUnit = "ABC"
    Department   = "XYZ"
  }
}

module "subnets" {
  source              = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=master"
  availability_zones  = ["${slice(data.aws_availability_zones.available.names, 0, var.max_availability_zones)}"]
  namespace           = "cp"
  name                = "jenkins"
  stage               = "prod"
  region              = "us-west-2"
  vpc_id              = "vpc-a22222ee"
  igw_id              = "igw-s32321vd"
  cidr_block          = "10.0.0.0/16"
  nat_gateway_enabled = "true"

  tags = {
    BusinessUnit = "ABC"
    Department   = "XYZ"
  }
}

Deploy Jenkins into a new VPC and new subnets

variable "max_availability_zones" {
  default = "2"
}

data "aws_availability_zones" "available" {}

module "jenkins" {
  source      = "git::https://github.com/cloudposse/terraform-aws-jenkins.git?ref=master"
  namespace   = "cp"
  name        = "jenkins"
  stage       = "prod"
  description = "Jenkins server as Docker container running on Elastic Beanstalk"

  master_instance_type         = "t2.medium"
  aws_account_id               = "000111222333"
  aws_region                   = "us-west-2"
  availability_zones           = ["${slice(data.aws_availability_zones.available.names, 0, var.max_availability_zones)}"]
  vpc_id                       = "${module.vpc.vpc_id}"
  zone_id                      = "ZXXXXXXXXXXX"
  public_subnets               = "${module.subnets.public_subnet_ids}"
  private_subnets              = "${module.subnets.private_subnet_ids}"
  loadbalancer_certificate_arn = "XXXXXXXXXXXXXXXXX"
  ssh_key_pair                 = "ssh-key-jenkins"

  github_oauth_token  = ""
  github_organization = "cloudposse"
  github_repo_name    = "jenkins"
  github_branch       = "master"

  datapipeline_config = {
    instance_type = "t2.medium"
    email         = "[e-mail]"
    period        = "12 hours"
    timeout       = "60 Minutes"
  }

  env_vars = {
    JENKINS_USER          = "admin"
    JENKINS_PASS          = "123456"
    JENKINS_NUM_EXECUTORS = 4
  }

  tags = {
    BusinessUnit = "ABC"
    Department   = "XYZ"
  }
}

module "vpc" {
  source                           = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=master"
  namespace                        = "cp"
  name                             = "jenkins"
  stage                            = "prod"
  cidr_block                       = "10.0.0.0/16"

  tags = {
    BusinessUnit = "ABC"
    Department   = "XYZ"
  }
}

module "subnets" {
  source              = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=master"
  availability_zones  = ["${slice(data.aws_availability_zones.available.names, 0, var.max_availability_zones)}"]
  namespace           = "cp"
  name                = "jenkins"
  stage               = "prod"
  region              = "us-west-2"
  vpc_id              = "${module.vpc.vpc_id}"
  igw_id              = "${module.vpc.igw_id}"
  cidr_block          = "${module.vpc.vpc_cidr_block}"
  nat_gateway_enabled = "true"

  tags = {
    BusinessUnit = "ABC"
    Department   = "XYZ"
  }
}

Makefile Targets

Available targets:

  help                                Help screen
  help/all                            Display help for all targets
  help/short                          This help short screen
  lint                                Lint terraform code

Inputs

Name	Description	Type	Default	Required
attributes	Additional attributes (e.g. policy or role)	list	<list>	no
availability_zones	List of Availability Zones for EFS	list	-	yes
aws_account_id	AWS Account ID. Used as CodeBuild ENV variable $AWS_ACCOUNT_ID when building Docker images. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html	string	-	yes
aws_region	AWS region in which to provision the AWS resources	string	us-west-2	no
build_compute_type	CodeBuild compute type, e.g. 'BUILD_GENERAL1_SMALL'. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html#build-env-ref-compute-types	string	BUILD_GENERAL1_SMALL	no
build_image	CodeBuild build image, e.g. 'aws/codebuild/docker:1.12.1'. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html#build-env-ref-available	string	aws/codebuild/docker:1.12.1	no
datapipeline_config	DataPipeline configuration options	map	<map>	no
delimiter	Delimiter to be used between name, namespace, stage, etc.	string	-	no
description	Will be used as Elastic Beanstalk application description	string	Jenkins server as Docker container running on Elastic Benastalk	no
env_default_key	Default ENV variable key for Elastic Beanstalk aws:elasticbeanstalk:application:environment setting	string	DEFAULT_ENV_%d	no
env_default_value	Default ENV variable value for Elastic Beanstalk aws:elasticbeanstalk:application:environment setting	string	UNSET	no
env_vars	Map of custom ENV variables to be provided to the Jenkins application running on Elastic Beanstalk, e.g. env_vars = { JENKINS_USER = 'admin' JENKINS_PASS = 'xxxxxx' }	map	<map>	no
github_branch	GitHub repository branch, e.g. 'master'. By default, this module will deploy 'https://github.com/cloudposse/jenkins' master branch	string	master	no
github_oauth_token	GitHub Oauth Token for accessing private repositories. Leave it empty when deploying a public 'Jenkins' repository, e.g. https://github.com/cloudposse/jenkins	string	``	no
github_organization	GitHub organization, e.g. 'cloudposse'. By default, this module will deploy 'https://github.com/cloudposse/jenkins' repository	string	cloudposse	no
github_repo_name	GitHub repository name, e.g. 'jenkins'. By default, this module will deploy 'https://github.com/cloudposse/jenkins' repository	string	jenkins	no
healthcheck_url	Application Health Check URL. Elastic Beanstalk will call this URL to check the health of the application running on EC2 instances	string	/login	no
image_tag	Docker image tag in the ECR repository, e.g. 'latest'. Used as CodeBuild ENV variable $IMAGE_TAG when building Docker images. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html	string	latest	no
loadbalancer_certificate_arn	Load Balancer SSL certificate ARN. The certificate must be present in AWS Certificate Manager	string	-	yes
loadbalancer_type	Load Balancer type, e.g. 'application' or 'classic'	string	application	no
master_instance_type	EC2 instance type for Jenkins master, e.g. 't2.medium'	string	t2.medium	no
name	Solution name, e.g. 'app' or 'jenkins'	string	jenkins	no
namespace	Namespace, which could be your organization name, e.g. 'cp' or 'cloudposse'	string	-	yes
noncurrent_version_expiration_days	Backup S3 bucket noncurrent version expiration days	string	35	no
private_subnets	List of private subnets to place EC2 instances and EFS	list	-	yes
public_subnets	List of public subnets to place Elastic Load Balancer	list	-	yes
security_groups	List of security groups to be allowed to connect to the EC2 instances	list	<list>	no
solution_stack_name	Elastic Beanstalk stack, e.g. Docker, Go, Node, Java, IIS. For more info: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html	string	64bit Amazon Linux 2017.09 v2.8.4 running Docker 17.09.1-ce	no
ssh_key_pair	Name of SSH key that will be deployed on Elastic Beanstalk and DataPipeline instance. The key should be present in AWS	string	``	no
stage	Stage, e.g. 'prod', 'staging', 'dev', or 'test'	string	-	yes
tags	Additional tags (e.g. map('BusinessUnit,XYZ)	map	<map>	no
use_efs_ip_address		string	false	no
vpc_id	ID of the VPC in which to provision the AWS resources	string	-	yes
zone_id	Route53 parent zone ID. The module will create sub-domain DNS records in the parent zone for the EB environment and EFS	string	-	yes

Related Projects

Check out these related projects.

• terraform-aws-elastic-beanstalk-applicationl - Terraform module to provision AWS Elastic Beanstalk application
• terraform-aws-elastic-beanstalk-environment - Terraform module to provision AWS Elastic Beanstalk environment
• terraform-aws-ecr - Terraform Module to manage Docker Container Registries on AWS ECR
• terraform-aws-efs - Terraform Module to define an EFS Filesystem (aka NFS)
• terraform-aws-efs-backup - Terraform module designed to easily backup EFS filesystems to S3 using DataPipeline
• terraform-aws-cicd - Terraform Module for CI/CD with AWS Code Pipeline and Code Build
• terraform-aws-codebuild - Terraform Module to easily leverage AWS CodeBuild for Continuous Integration

Help

Got a question?

File a GitHub issue, send us an email or join our Slack Community.

Commercial Support

Work directly with our team of DevOps experts via email, slack, and video conferencing.

We provide commercial support for all of our Open Source projects. As a Dedicated Support customer, you have access to our team of subject matter experts at a fraction of the cost of a full-time engineer.

• Questions. We'll use a Shared Slack channel between your team and ours.
• Troubleshooting. We'll help you triage why things aren't working.
• Code Reviews. We'll review your Pull Requests and provide constructive feedback.
• Bug Fixes. We'll rapidly work to fix any bugs in our projects.
• Build New Terraform Modules. We'll develop original modules to provision infrastructure.
• Cloud Architecture. We'll assist with your cloud strategy and design.
• Implementation. We'll provide hands-on support to implement our reference architectures.

Community Forum

Get access to our Open Source Community Forum on Slack. It's FREE to join for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build sweet infrastructure.

Contributing

Bug Reports & Feature Requests

Please use the issue tracker to report any bugs or file feature requests.

Developing

If you are interested in being a contributor and want to get involved in developing this project or help out with our other projects, we would love to hear from you! Shoot us an email.

In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.

1. Fork the repo on GitHub
2. Clone the project to your own machine
3. Commit changes to your own branch
4. Push your work back up to your fork
5. Submit a Pull Request so that we can review your changes

NOTE: Be sure to merge the latest changes from "upstream" before making a pull request!

Copyright

Copyright © 2017-2018 Cloud Posse, LLC

License

See LICENSE for full details.

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.

Trademarks

All other trademarks referenced herein are the property of their respective owners.

About

This project is maintained and funded by Cloud Posse, LLC. Like it? Please let us know at [e-mail]

We're a DevOps Professional Services company based in Los Angeles, CA. We love Open Source Software!

We offer paid support on all of our projects.

Check out our other projects, apply for a job, or hire us to help with your cloud strategy and implementation.

Contributors

Andriy Knysh	Ivan Pinatti	Sergey Vasilyev