larsthorup / docker-sandbox

Deployment engineering with Docker

Development environments and delivery pipelines for demo and live environments

environment	cluster	orchestrator	reg	dns	ssl	app	api	mig
dev	local	compose	-	-	-
local	local	compose	-	-	-	+	+	+
demo	local	swarm	+	+	+	+	+	+
demo	AWS EC2	swarm		+
live	AWS ECS	copilot		+	+	+	+
local	local	kubernetes
live		kubernetes

• Local deploy (Docker Compose)
• Local deploy (Docker Swarm)
• Demo deploy (Docker Swarm)
• Local deploy (Kubernetes)
• Live deploy (Kubernetes)
• Live deploy (AWS Copilot)

Services of the application

• App (HTML web UI)
• API (HTTP web service)
• Migrator (SQL scripts)

Supporting services

• Docker: stacks, containers, networking, logs, secrets, configs
• PostgreSQL: database
• Traefik: proxy, load balancing, HTTPS
• Docker Registry: container images
• DNS: AWS
• Virtual Machines: AWS
• mkcert: SSL for local environment
• Let's Encrypt: SSL for public (demo and live) environments

Environments:

• dev (local with watch mode)
• local
• demo
• live

Orchestrators:

• Physical hardware (no containerization)
• Single docker image
• Docker Compose
• Docker Swarm
• Kubernetes
• AWS Copilot

Hosting providers:

• On premise
• AWS
• Google
• Azure

Scripting language:

• Bash, YML, Dockerfile, .env
• AWS Cloudformation
• Terraform

docker-compose up --build

TODO

• deploy demo environments on cloud hosted docker swarm on https://${STACK}.${SWARM_HOST}
  • with a registry for demo environments?
    • REGISTRY_HOST
    • TAG=latest
    • docker-compose build
    • docker-compose push
    • ssh ${SWARM} "docker stack deploy"
    • docker image prune
    • https://gabrieltanner.org/blog/docker-registry
    • https://semaphoreci.com/community/tutorials/running-applications-on-a-docker-swarm-mode-cluster
    • https://geek-cookbook.funkypenguin.co.nz/ha-docker-swarm/registry/
  • build on demo
    • ssh ${SWARM} "docker-compose build"
    • not concurrency safe
  • without a registry
    • docker stack deploy ${SWARM}
    • does not appear to be an option??
• move /.env to compose/.env
• swarm/.env: GREETER_HOST=greeter.xpqf.net
• registry: authentication
• traefik: authentication
• watch mode for Docker
  • https://stackoverflow.com/questions/26050899/how-to-mount-host-volumes-into-docker-containers-in-dockerfile-during-build
  • https://vsupalov.com/rebuilding-docker-image-development/
  • https://vsupalov.com/cache-docker-build-dependencies-without-volume-mounting/
• Dev (watch) mode for app
• Dev (watch) mode for api (nodemon?)
• CI/CD Pipeline
  • TAG=${COMMIT_SHA}
  • docker-compose build
• e2e test of deployment
• EC2: Amazon Linux (over Ubuntu Linux) for better Cloudformation integration
• docker: view logs also after container has crashed
• traefik: route tcp from db.${STACK}.${SWARM_HOST} to service
• pg: wait until healthy: https://stackoverflow.com/questions/35069027/docker-wait-for-postgresql-to-be-running
• traefik: up.sh: wait until healthy
• registry: up.sh: wait until healthy
• avoid "WARNING: Some services (traefik) use the 'configs' key, which will be ignored."
  • use another template mechanism than docker-compose config
• swarm: swarmpit - resource dashboard
• swarm: swarmprom - monitoring and alerts
• swarm: swarm deployment dashboard - https://github.com/dockersamples/docker-swarm-visualizer
• prettify: yaml
• linting of infrastructure code files (docker-compose config)
  • https://github.com/replicatedhq/dockerfilelint
  • https://github.com/koalaman/shellcheck
• DB for copilot
• Migration for db
• live telemetry
• Recreate db command
• devcontainer.json for vscode
• app: vite bundler
• Metrics for build time / deploy time / watch CPU usage
• Deploy to Kubernetes
• can we use envsubst more?