hafizpatwary / microservices-country-quiz

An application that takes advantage of micro-service architecture and containerisation

YeezyJet Country Quiz

A flag quiz, that rewards the user with a 'voucher' for a plane ticket.

Prerequisites

• Git
• Docker
• Docker Compose (Will already be installed if you are using MacOS or Windows) click here for Linux

Requisites for application

• In order for the application to work you require an API key from openweathermap, it is free to sign up and the service used for this application is free. Important: Do not upload your API key online.
• A MySQL instance with a database set up

Setup

Open up a terminal and run the commands shown below to get setup.

Clone the Repository

$ git clone https://github.com/hafizpatwary/microservices
$ cd microservices

Configure MySQL database

Note: SECRET_KEY can be set to any arbitrary string

$ export MYSQL_USERNAME=[<YOUR_MYSQL_USERNAME>]
$ export MYSQL_PASSWORD=[<YOUR_MYSQL_PASSWORD>]
$ export MYSQL_IP=[<YOUR_MYSQL_IP>]
$ export SECRET_KEY=[<YOUR_SECRET_KEY>]

Configure API key

$ export API_KEY=[<YOUR_API_KEY>]

Set up databse

$ bash db_setup.sh

Build & Run the Container

docker-compose up -d --build

Check containers running

Once docker-compose completes the build

$ docker ps -a

Output should show running containers:

CONTAINER ID        IMAGE                                     COMMAND                  CREATED             STATUS              PORTS                NAMES
1e218274632a        nginx                                     "nginx -g 'daemon of…"   18 minutes ago      Up 18 minutes       0.0.0.0:80->80/tcp   nginx
f7f69a48f2e9        jenkins:5000/frontend_service:build-      "/usr/local/bin/pyth…"   18 minutes ago      Up 18 minutes                            microservices_frontend_1
ec1aedfc58d2        jenkins:5000/prize_service:build-         "/usr/local/bin/pyth…"   25 minutes ago      Up 25 minutes                            microservices_prize_1
8b5183c4888f        jenkins:5000/countries_service:build-     "/usr/local/bin/pyth…"   25 minutes ago      Up 25 minutes                            microservices_countries_1
af5dfccc0a06        jenkins:5000/temperature_service:build-   "/usr/local/bin/pyth…"   25 minutes ago      Up 25 minutes                            microservices_temperature_1

Check web application working

Open your browser and go on http://localhost:80 or to check if website is up in terminal with:

$ curl localhost

Should see this:

Remove containers and images

$ docker-compose down
$ docker image rm $(docker image ls -q) # !This will destroy all images stored in your machine

Microservices

An application that takes advantage of micro-service architecture and contenarisation

Index

Brief

• Solution
• Architecture
• Micorservices
• Entity Relationship Diagrams

Risk Assessment

• Risk Sssessment Table
• Explanations

Deployment

• Technologies Used
• Testing

Project planning and user stories

• Use case diagram
• UML diagram

Improvements for the Future Authors

The Brief

The aims of the project are as follows:

• Create a service ortiented web-application, that is comprised of at least four services. Each service must have two different implementation that can be swapped between them without disrupting the user experience.
• Build a CI pipeline using Jenkins that automatically updates and deploys the web-app when source code is updated.
• Create ansible playbook to automate configuation of resources (VMs) required for this project

Solution

For my web application I decided to create a quiz game. The app presents the user a randomly generated flag, where the user has to guess which country the flag belongs to based on a multiple choice question. If the user guesses correctly, the user is rewarded with a voucher for a plane ticket.

Architecture

To create the app, the services architecture is shown below:

Country generator service:

• This service essentially generates the question that will be shown on the frontend. To create the multiple choice question, it generates a random list of unique countires from a json file. Once the selection has been done, it parses a json package with the countries and a flag from one of those countries.

Temperature api service:

• This service selects a random city from a list of cities and makes an api call to openweathermap.com to get the current temperature. The response is decoded and then parsed to the desired JSON format to be used by service 4.

Voucher generator service:

• Combines the JSON objects recieved from service 2 and 3 with some logic to create a voucher. The current logic set is to generate a prize based on number of options, set in the quiz, and the current weather.

Frontend service:

• This is the service that the user will be interacting with. The frontend uses the country service to retrieve the flag quiz. Upon submission of quiz, if the answer is correct, the forntend then proceeds to make a get request to the voucher service to reward the user. Additionally it also persist some data to an SQL database, such as the response of the user to the question and user's email address used to retrieve the voucher.

Entity Relationship Diagrams

Altough having an entity relationship diagram was not a requirement for this project, it was useful to revise some database concepts and practice SQLAlchemy's syntax. The tables above are used by the frontend to store the answer to the quiz given by the user and email address given to retrieve the voucher.

Risk Assessment

Risks	Likelihood	Impact	Explanation
Uploading API key to GitHub	High	High	1 Click here
External database manipulation	Medium	High	2 Click here
Automation causing issues	Medium	High	3 Click here
Container losing data when VM stopped	High	High	4 Click here
Website malfunctions because code is broken	Medium	High	5 Click here

Uploading API key to GitHub

Service three retrieves the temperature of a random city by making an api request to openweathermap.com. The website lets the application retrieve the data required for this project for free, however there also paid services which requires the API key. When writing source code I might upload these keys on GitHub my mistake: Solution:

• Set the API key as an enviorment variable

External database manipulation

During development it is likely that I will be working on different machines, hence there will be a public Git repo. It is very likely that I may upload some credentials by mistake

Solution:

• Set enviornmental variables so that credentials can be accesed by one person only
• Delete credentials if you know someone else might use the same machine

Automation causing issues

Automation can save a lot of time and hassle if done right, however if not done properly it can:

• When writing Ansible playbook, it is important to write them as versatile as possible, like not having the host name hardcoded but instead one that varies automatically.

Solution:

• Automate only repetitive tasks such as deployment
• Do not automate a task that is not repetitive, such as setting environmental variables

Container losing data when VM stopped

It is important to realise that when a VM is stopped, the docker containers exit. On restart of VM, the containers can also be restarted, however they won't be the same as before. Solution:

• Set container to the option "--restart=always"
• Set volumes to the container

Website malfunctions because code is broken

• Always test before pushing code
• Set up a test enviornment

Deployment

The test and deployment process for the web app was automated using Jenkins, a CI server. Jenkins runs in a GCP instance that automatically deploys the webapp into deployment server, with a webhook to GitHub which is triggered with every push event. Ansible was used to create VMs with the correct configuration, to automate the process of setting environmetns for production and CI server.

Jenkins job:

• Once code is pushed to GitHub, Jenkins is triggered
• Jenkins downloads executable Jenkinsfile
• It run an automated test on the code
• Build images from yaml file and pushes image to local repo
• Pulls image down on Deployment server and makes a rolling update.

Technologies Used

• Database: GCP SQL Server
• Programming language: Python
• Framework: Flask
• Deployment: Gunicorn
• CI Server: Jenkins
• Test Reporting: Pytest
• VCS: Git
• Project Tracking: Trello
• Live Environment: GCP
• Containerization: Docker
• Configuration Management: Ansible
• Orchestration: Docker-compose

Testing

Testing has been done using pytest. The coverage report for the services overall is 92%. The breakdown of test coverage for each service is reported below with explanation wherever is necessary.

Test for Service 2 (Contry Genarator)

Test for Service 3 (Temperature)

Test for Service 4 (Vouhcer Generator)

Test for Service 1 (Frontend)

Project planning and user stories

Project tracking was done using a trello board. Below the before and after of the project.

Improvements for the Future

• Currently the job running on Jenkins takes about two minutes, to run. Time can be reduced by about 10 seconds, by making jenkins cloning down only what is necessary for deplyment. Hence, ignoring file such as documentation, images and other unused code.
• In Ansible configuration I would break down the task in smaller files and making them more generic so that I can use them in the future for other projects.
• I would make an SQL container, rather than using a production ready database from GCP. I would save time during development and save gcp credit