- Devops Challenge
- Evaluation Criteria
- Architecture
- Phase 1 ( How to run the project Locally using Docker)
- Phase 2 ( Containerizing the 3tier web app Using Docker)
- Phase 3 ( Building CI to push the 3tier web app to dockerhub using Jenkins)
- Phase 4 ( Deploying the 3tier web app with Kubernetes using Deployment, Services and Ingress)
- Phase 5 ( Deploying the 3tier web app with Helm Charts)
- Phase 6 ( Building CD of the 3tier web app Using ArgoCD and Helm Charts)
- Phase 7 ( How to test the 3tier web app from anywhere )
- Time for some Screen Shots
You have been tasked with deploying a new application in a containerized environment using Docker and Kubernetes.
Your task is to design and implement a solution that meets the following requirements:
- Setup a sample web application. For example, a basic PHP web app using Laravel.
- The application must have 3 or more components. For example: A database, a cache service, and a backend.
- Containerize it using Docker.
- Use Kubernetes to manage the lifecycle of the containers, including deployment, scaling and management.
- Deploy the application using Helm.
- Implement continuous integration and delivery pipelines. Use the tool of your choice. Using Gitlab CI is a plus. Expected Outcome:
- A design diagram for the application and how it is distributed inside your cluster.
- Docker files, CI/CD configuration, Helm chart, and Kubernetes manifests.
- A README that contains:
- How to deploy the application. Including all the steps needed to have the application up and running on a Kubernetes cluster.
- A listing of the assumptions made and how you prioritized the requirements.
- A summary of the testing process, including any issues encountered and how they were resolved.
The evaluation will be based on the completeness and quality of the solution, including the following criteria:
- Correctness: Does the solution meet all the requirements listed above?
- Quality: Well-organized, easy to read, and maintainable.
- Documentation: Is there clear and concise documentation provided for the solution, including instructions for installation and troubleshooting?
- A user opens the
frontend
onlocalhost:3000
- clicks on
ADD Todo
and a small window will pop up asking for the title of todo item, a descrption and the due data - and once the user clicks on
Save
a request is sent to theapi backend
- which then stores the data in a
mongo db
- and the saved info gets displayed to the user.
- Make sure you have Docker installed and running using their official website.
- At the command prompt simply type
docker-compose up
- Docker will then create the MongoDB from the stock mongo image. The api uses nodejs with express and is built from a node:alpine image. The front end uses ReactJS and built from a node:alpine image.
-
You can build a docker image using this command and tag it using the
-t
docker build -f ./frontend/Dockerfile -t ahmedsoliman202/3tier-frontend-app . docker build -f ./backend/Dockerfile -t ahmedsoliman202/3tier-backend-app .
-
Before running the
backend-app
andfrontend-app
containers, We have to- 1- create a network to be used by the 3 containers using this command
docker network create mongodb
- 2- run a db container first, passing all the required Environment Variables, voluem, network and right ports
docker run -d --name mongo -p 27017:27017 -e MONGO_INITDB_ROOT_USERNAME=username -e MONGO_INITDB_ROOT_PASSWORD=password -v mongodb_data:/data/db --network mongodb mongo
-
After building the
backend-app
andfrontend-app
images, it is time to test them, passing the required Environment Variables, network and right portsdocker run -d --name backend -p 3001:3001 --network mongodb 3tier-backend-app docker run -d --name frontend -p 3000:3000 --network mongodb 3tier-frontend-app
-
Wrote a
docker-compose.yml
file to eaisly spin up the 3 tier web app containers all together with one command- if you are using a
Unix/Linux
OS then docker-compose doesn't come installed with Docker so you have to install it manually from here - You can run a docker compose file using the following command
docker-compose up
- if you are using a
-
Side Note:
- once the backend app starts, it tries to connect to the db, that's why we launch the db container first but in case of docker-compose. both backend-app and db containers start immediately, and the db container haven't loaded yet so the backend-app crashes and to fix this issue, I implemented the following approache:
- Added a
depends_on
tag in the backend-app section to force the container to wait till the db container is up and running and then start the backend container
- Added a
- I used -f with
docker build
to specifiy the location of Dockerfile and.
to specify the location which includes the files Dockerfile deals with
- once the backend app starts, it tries to connect to the db, that's why we launch the db container first but in case of docker-compose. both backend-app and db containers start immediately, and the db container haven't loaded yet so the backend-app crashes and to fix this issue, I implemented the following approache:
-
Make sure to have Jenkins installed using their website here .
-
You have to rename the docker image to include you dockerhub username
- You can rename a docker image using this command
docker tag 3tier-backend-app:latest ahmedsoliman202/3tier-backend-app:latest docker tag 3tier-frontend-app:latest ahmedsoliman202/3tier-frontend-app:latest
- You can push a docker image to dockerhub using this command
docker push ahmedsoliman202/3tier-backend-app:latest docker push ahmedsoliman202/3tier-frontend-app:latest
-
Backend created image can be found here backend-image:latest
-
Frontend created image can be found here frontend-image:latest
-
Side Note:
- in order to use Jenkins to build and push docker images the host pc must have docker installed and in my case I accessed the Jenkins container and mounted docker socket volume into the Jenkins container to enable the execution of Docker commands on the host machine
- another nice alternative is using this docker image which has light Jenkins image
docker run -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock liatrio/jenkins-alpine
-
Make sure you have
kubectl
,minikube
installed from the Kubernetes docs- You also must have a
minikube driver
installed, in my case I usedDocker Desktop
- In order to interact with the minikube cluster you have have to start it using this command
minikube start
- You also must have a
-
Since the backend-app depends on the db, We have to apply db kubernetes manifest first using this command
k create -f ./kubernetes_mainfests/db-deployment.yaml
- To see if the pod, pv, pvc were created successfully and running, using this command to list all the pods in a cluster node
kubectl get pods kubectl get pv kubectl get pvc
- To see if the pod, pv, pvc were created successfully and running, using this command to list all the pods in a cluster node
-
Second step is to deploy the web app using this command
k create -f ./kubernetes_mainfests/backend-deployment.yaml k create -f ./kubernetes_mainfests/frontend-deployment.yaml
-
I defined a k8
Service
resource inbackend-deployment.yaml
andfrontend-deployment.yaml
of typeNodePort
so that I can Expose the web app to the public- In order to access the web app publicly, We have to get the cluster ip using this command
minikube ip
- I have set the
NodePort
to30008
so we can use the cluster ip along with the nodeport to test the app, The url will be something likehttp://192.168.197.128:30008
- Another approach will be to ask minikube to expose the service and open it in the browser using the follwing command
minikube service frontend-app-svc
- In order to access the web app publicly, We have to get the cluster ip using this command
-
I defined a k8
Ingress
resource iningress.yaml
of path typePrefix
and typeSimple Fanout
so that I can Expose the web app to the public and access it using human readable host likerobusta.io
and ingress will take care of routing the traffic based on theURI
to the right servies and hence to the right pods-
In order to deploy the ingress manifest, We can use this command
k create -f ./kubernetes_mainfests/ingress.yaml
-
Now we need to add a rule in minikube to route the traffic using minikube ip (
192.168.49.2
) to the host (robusta.io
) which we added in the ingress as follows:minikube ssh sudo /bin/sh -c 'echo "192.168.49.2 robusta.io" >> /etc/hosts'
-
to test the web app, within the minikube ssh session we can query the URLS of our 3tier web app
curl robusta.io/ curl robusta.io/backend curl robusta.io/db
-
Side Note:
- I have set the
replicas
in thebackend-deployment.yaml
andfrontend-deployment.yaml
to5
to achieve high availability of the service. - I have defined a
PersistentVolume
resource indb-deployment.yaml
to achieve volume persistence.
- I have set the
-
Make sure to have Helm installed from official docs.
-
You can create a
Helm Chart
using this commandhelm create chartnamehere
-
You can verify the helm charts before deploying by generating the template using this command
helm template --output-dir ./charts-test-templates --values=./helm_charts/backend-app/values.yaml ./helm_charts/backend-app/
-
Like we did in kubernetes, We will deploy db first using this command
helm install mongo --values=./helm_charts/db/values.yaml ./helm_charts/db
-
Then we deploy web app using this command
helm install backend-app --values=./helm_charts/backend-app/values.yaml ./helm_charts/backend-app/ helm install frontend-app --values=./helm_charts/frontend-app/values.yaml ./helm_charts/frontend-app/
-
Side Note:
- I have defined a
HorizontalPodAutoscaler
resource inhpa.yaml
to achieve autoscaling and high availability of the service.
- I have defined a
-
First thing to do before installing Argocd is to create a kubernete namespace for argocd using this command
kubectl create namespace argocd
-
Make sure to have ArgoCD installed from official docs.
- You can get the Url where Argocd is hosted, it will be something like
http://192.168.197.128:30196
by using this commandminikube service argocd-server -n argocd --url
- Default username is
admin
and the password can be loacated using this commandkubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -
- You can get the Url where Argocd is hosted, it will be something like
-
To interact with Argocd, You have to install ArgoCD CLI:
- from ArgoCD CLI official docs
- Or you can download the release from their github release page and make sure the version you installed matches the release version.
- After installing argocd cli, You need to login because the cli doesn't know where to find the instance running argocd
- Once Argocd Cli is installed, We login to Argocd using the service url from above by using this command
argocd login 192.168.197.128:30196
- A good place to find reliable examples on how to use the CLi, creating applications, Using UI, can be found Offcial Docs
-
Like we did in kubernetes & Helm We will deploy db first using this command
argocd app create -f ./argocd_files/db-mainfest.yaml
-
Then we deploy web app through Argocd Cli using this command
argocd app create -f ./argocd_files/backend-mainfest.yaml argocd app create -f ./argocd_files/frontend-mainfest.yaml
-
We can access the web app, it will be something like
http://192.168.197.128:30008
using this commandminikube service backend-app-svc -n argocd --url
-
Side Note:
- If you are using windows, make sure to rename the downloaded argocd cli from
argocd-windows-amd64.exe
toargocd.exe
- Aslo make sure to add
argocd.exe
location to windows environment variablepath
- If you are using windows, make sure to rename the downloaded argocd cli from
- I'm hosting the containerized 3tier web app on an AWS EC2 instatnce.
- Frontend app can be accessed through this URL:
http://http://52.10.23.104:3000
- Backend app endpoint can be accessed through this URL:
http://http://52.10.23.104:3001/api
- Running the web app locally
- Testing The app while it's running through docker-compose
- Building a Docker image
- Starting backend-app container and db-container through docker-compose
- Running Jenkins stages to build and push the docker image to Dockerhub
- Jenkins successfully built and pushed the docker image to Dockerhub
- Docker image repo on Dockerhub
- Deploying web app and db on kubernetes using minikube cluster
- Deploying backend-app and db on kubernetes using Helm charts