kosicki123 / awesome-ecs

A curated list of awesome ECS guides, development tools, and resources

A curated list of guides, development tools, and resources for Amazon Elastic Container Service (ECS). This list includes both community created content as well as content created by AWS.

Want to add something? Open a PR! 🙂

Pick your container hosting strategy:

• AWS Fargate - AWS Fargate is a technology for Amazon ECS that allows you to run containers without having to manage servers or clusters.
• Self hosted in EC2 - Running your own cluster of EC2 instances to host your containers gives you the most control over price (ability to run on spot instances or reserved instances) as well as configuration.

AWS Fargate

Setting up Fargate

• fargate - Command line tool for interacting with AWS Fargate. With just a single command you can build, push, and launch your container in Fargate, orchestrated by ECS.
• CloudFormation Templates - Sample CloudFormation templates to help you spin up a Fargate cluster, and service in that cluster automatically

Application Deployments in Fargate

• Deploying a Rails app to Fargate - Step by step walkthrough of deploying a Ruby + RDS app, with helpful debugging tips
• How to use AWS Fargate and Lambda for long-running processes in a Serverless app - Great tutorial showing how to leverage the power of long running docker containers in Fargate alongside Lambda. Example application processes video files to extract thumbnails using just S3, Lambda, and Fargate... no EC2.

Self hosted in EC2

• Getting Started
• Automated Infrastructure
• Build and Deploy Tools
• Implementation Guides
• Open Source Tools
• Reference Architectures
• Continuous Integration / Continuous Deployment
• Presentations
• Tech Blogs

Getting Started

• The Hitchhikers Guide to AWS ECS and Docker by J. Cole Morrison - Introduction to AWS ECS concepts
• Working with AWS ECS by Sriram Rajan of Rackspace

Automated Infrastructure

Tools to help you set up a cluster of EC2 hosts for ECS to use to run your docker containers:

• CloudFormation ECS - Reference architecture for deploying microservices to ECS in tiered VPC with NAT gateways and two availability zones.
• Terraform ECS by Armin Coralic - Production ready AWS ECS infrastructure as code with Terraform
• CloudFormation Templates by Cloudonaut
• mu - This devops tool helps you setup ECS as a platform to deploy microservices on
• Deployfish by Caltech ADS - Manage your whole application lifecycle in Amazon ECS
• ecsq - A developer friendly tool for querying the state of an ECS cluster

Build and Deploy Tools

Tools to help you interact with ECS to launch your containers on your cluster of self managed EC2 instances:

• coldbrew - Fantastic tool that provisions ECS infrastructure, builds and deploys your container, and connects your services to an application load balancer automatically
• ecs-cli - Docker Compose compatible deployment tool by AWS
• empire - Control layer on top of ECS that provides a Heroku like workflow
• broadside - Ruby based command line tool for deploying to ECS
• uFO - Ruby based tool for building containers and shipping them to ECS
• bash deployment script by Justin Kulesza
• pnzr - Go based tool for building and pushing to ECS, also has integraton with AWS KMS for secrets management.
• deplojo - Python based deployment tool using ECS
• convox - Easily build, deploy and scale applications on ECS
• ecsctl - Open source tool similar to Kubernetes kubectl for ECS.
• ecs-deploy - Simple but powerful tool for initiating automatic blue green deploys on ECS
• ecspresso - Minimalistic: JSON file goes in, service launches

Implementation Guides

Examples of how to do advanced customizations on your ECS cluster:

• Autoscaling
  • Autoscaling services in ECS - How to autoscale the number of service tasks
  • Autoscaling the cluster in ECS - How to autoscale the number of EC2 instances in a cluster
  • Advanced ECS cluster autoscaling - A more advanced technique for autoscaling an ECS cluster, with more effective scale in as well as out
  • Lambda based approach for cluster scale-in
  • Extending ECS autoscaling for under $2 a month with Lambda
  • Scale in script in Python
• Networking
  • Private subnets tutorial
• Service Discovery
  • Using DNS
  • Using Consul
  • Using Weaveworks
  • Using linkerd
• Data Persistance
  • Using Amazon Elastic File System
• Secrets Management
  • Using Parameter Store and IAM Roles for Tasks
  • Using ECS Task roles for managing AWS credentials
  • The right way to store secrets using Parameter Store
• Administration
  • Automatically draining cluster instances before termination
  • Monitor cluster state with Cloudwatch event stream
  • Run an ECS Task on every cluster instance
• Docker for .NET Developers
  • Part One: Intro
  • Part Two: Our First dockerfile

Other Open Source Tools

• Blox - Framework for advanced cluster management and scheduling
• Watchbot - This tool by Mapbox helps you run data processing across an ECS cluster in response to external events
• ecs-export - A tool for exporting ECS cluster metrics to Prometheus for advanced querying
• docker-elk-ecs - Connecting Amazon ECS container logs to an ELK (Elasticsearch, Logstash, Kibana) stack
• Sample task definitions - Sample task definitions for running applications like Nginx, Tomcat, Gunicorn, Wildfly, Kibana, and Jetty as containers under Amazon ECS

Reference Architectures

• Microservice cloudformation stack
• Sock shop microservices demo on Amazon ECS
• Node.js Microservices
• Java Microservices on AWS ECS
• Swift ECS Workshop
• NGINX Reverse Proxy sidecar container on AWS ECS
• Deploying a Deep Learning Framework on ECS
• Powering your Amazon ECS Cluster with Amazon EC2 Spot Instances
• Reactive Microservices Architectures with Amazon ECS, AWS Lambda, Amazon Kinesis Streams, Amazon ElastiCache, and Amazon DynamoDB

Continuous Integration / Continuous Deployment

• CI/CD using:
  • AWS CodePipeline
  • Atlassian Bamboo
  • Lambda
  • CircleCI
  • Shippable
  • Gitlab
  • SemaphoreCI
• CI/CD mechanics:
  • Rolling blue/green deploy in place
  • Canary Deploy
  • Isolated Regression Testing

Presentations

• Running your Dockerized application(s) on AWS EC2 Container Service by Marco Pas
• Microservices on AWS with Weaveworks
• Running a Virtual World via ECS - Linden Labs on their usage of ECS
• Advanced Task Scheduling with AWS ECS
• Instacart on running microservices on Amazon ECS
• Building Next-Generation Applications with Amazon ECS - How Meteor Built Galaxy on Amazon ECS
• Amazon ECS at Coursera: A General Purpose Microservice

Tech Blogs

• Airtime - Microservice Continuous Integration Made Easy with AWS ECS
• Segment - Rebuilding Our Infrastructure with Docker, ECS, and Terraform
• Nextdoor - How Nextdoor made a 10x improvement in release times with Docker and Amazon ECS
• MapBox - We switched to Amazon ECS and you won't believe what happened next
• Wrapp - On how EC2 Container service simplified our infrastructure stack
• Docker on AWS: from containerization to orchestration

Courses

• Microservices with Docker, Flask, and React - Learn how to build, test, and deploy microservices powered by Docker, Flask, React Amazon ECS!