danitfk/service-example

Flask App + RabbitMQ + PostgreSQL + Docker

The goal of this project was creating a Docker-Compose file to have these services:

Database (PostgreSQL)
Messaging queue (RabbitMQ)
HTTP App (Python app based on Flask + uWSGI)
Async Worker (Python, Celery)
NGINX Serving the HTTP app and its static files

Each of above services run by separated docker container.

🔨 Requirements

Installed docker,docker-compose, git (Docker Engine release +v1.13.0) [HOW TO]
Available HTTP/HTTPS port on Docker machine
Proper Internet connectivity

✔️ Index

Clone source code and docker-compose
Adjust environment variables
Generate self-signed SSL certificate (optional, already included)
Generate Diffie-Hellman group (dhparam) (optional, already included)
Run service compose file

⚙️ Run services

Clone source code and docker-compose

First make a clone from this repository into your local machine, Which docker and docker-compose installed before.

git clone https://github.com/danitfk/service-example

Adjust environment variables

You can define some credentials related to PostgreSQL and RabbitMQ service in .env file like bellow:

POSTGRES_PASS=postgrepass
POSTGRES_USER=postgreuser
POSTGRES_DB=postgredb
RABBITMQ_DEFAULT_USER=rabbituser
RABBITMQ_DEFAULT_PASS=rabbitpass
RABBITMQ_DEFAULT_VHOST=rabbitvhost

Generate a Self Signed SSL Certificate

This part is optional, SSL Certificate files already included in certs directory.

You have to create SSL Certificate related files in certs directory like this:

First We will create a Self Signed SSL Certificate with OpenSSL. Make sure OpenSSL package installed on your system (How to install openssl)

# Create certs directory if not exists
mkdir certs
# Generate a self-signed key and certificate pair with OpenSSL
openssl req -x509 -nodes -days 365 -newkey rsa:4096 -keyout certs/nginx-selfsigned.key -out certs/nginx-selfsigned.crt

The entirety of the prompts will look something like this:

Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:New York
Locality Name (eg, city) []:New York City
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Bouncy Castles, Inc.
Organizational Unit Name (eg, section) []:Ministry of Water Slides
Common Name (e.g. server FQDN or YOUR name) []:server_IP_address
Email Address []:admin@your_domain.com

Generate Diffie-Hellman group

This part is optional, dhparam file already included in certs directory.

It may takes a while,to reduce time change 4096 to 2048.

openssl dhparam -out certs/dhparam.pem 4096

Run docker compose file

First, make sure your system already has docker-compose then run docker-compose command:

docker-compose up -d

After download and build images, the output should be like this:

Starting serviceexample_broker_1 ... 
Starting serviceexample_broker_1
Starting serviceexample_db_1 ... 
Starting serviceexample_db_1 ... done
Starting serviceexample_app_1 ... 
Starting serviceexample_broker_1 ... done
Starting serviceexample_worker_1 ... 
Starting serviceexample_app_1 ... done
Starting serviceexample_proxy_1 ... 
Starting serviceexample_proxy_1
Starting serviceexample_migration_1 ... 
Starting serviceexample_proxy_1 ... done

📓 Notes:

Explain more about the project including changes which are made.

Some code changes in the source

To retrieve BROKER_URL in worker or DATABASE_URL in App I've done some code changes to get those variable from environment. (worker.py and alembic env.py)

🛡️ NGINX Security

There are some best practices to secure an NGINX Web Server which publicly is available.

I did implement the important thing of NGINX security in Dockerized environment such as:

Limit concurrent requests per IP
Remove Server tokens (nginx version)
Limit available methods (GET,POST,HEAD)
Controlling buffer overflow attacks
NGINX SSL Configuration (Using strong ciphers which Mozilla suggested, Disable old and vulnerable TLS version, Strong dhparam, etc.. )
Avoid clickjacking with X-Frame-Options SAMEORIGIN
Disable content-type sniffing on some browsers
Enable the Cross-site-scripting (XSS) filter with X-XSS-Protection "1; mode=block"
Force HTTPS

🛡️ Docker Security

Non-root user for application

Based on best practices applications such as app and migration will run with a non-root user.

Separated Network

I've applied two separated network app which applications' container such as app, migration, broker, worker, db connects to each other only in Internal network and another separated network web which proxy and app connect to each other.

🤔 Better option for caching

As mentioned before in "DevOps Engineer assignment", caching in NGINX has to be done with Mircocaching but there is a better option for caching.

Serve static content (Images/styles/javascripts/...) directly on NGINX

It can be done through mounting static directory of the flask as ReadOnly volume in NGINX and serve those file directly on NGINX and neither of requests will pass to the backend (uWSGI) in this case.

With the current setup, Caching will be done with NGINX Microcache but there can found configuration for serving content directly with expires headers in nginx.conf commented.

danitfk / service-example