Conferences and conventions are hotspots for making connections. Professionals in attendance often share the same interests and can make valuable business and personal connections with one another. At the same time, these events draw a large crowd and it's often hard to make these connections in the midst of all of these events' excitement and energy. To help attendees make connections, we are building the infrastructure for a service that can inform attendees if they have attended the same booths and presentations at an event.
You work for a company that is building an app that uses location data from mobile devices. Your company has built a Proof of concept (POC) application to ingest location data named UdaConnect. This POC was built with the core functionality of ingesting location and identifying individuals who have shared close geographic proximity.Below is the UdaConnect App diagram:
Management loved the POC, so now that there is buy-in, we want to enhance this application. You have been tasked to enhance the POC application into a Minimum Viable Product (MVP) to handle the large volume of location data that will be ingested.
To do so, you will refactor this application into a microservice architecture using message passing techniques that you have learned in this course
-
- Setup The Environment Install the tools to get our environment set up properly. If you haven't already done so:
- Install Docker
- Set up a DockerHub account
- Set up kubectl
- Install VirtualBox with at least version 6.0
- Install Vagrant with at least version 2.0
-
- Initialize K3s To run the application, you will need a K8s cluster running locally and interface with it via kubectl. We will be using Vagrant with VirtualBox to run K3s.
In this project's root, run vagrant up. 'vagrant up' The command will take a while and will leverage VirtualBox to load an openSUSE OS and automatically install K3s. Some useful vagrant commands can be found in this cheatsheet. A good introduction to Vagrant can be found in this article.
-
- Retrieve the Kubernetes config File After vagrant up is done, you will SSH into the Vagrant environment and retrieve the Kubernetes config file used by kubectl. We want to copy this file's contents into our local environment so that kubectl knows how to communicate with the K3s cluster.
We want to copy this file's contents into our local environment so that kubectl knows how to communicate with the K3s cluster.
-
- Configure kubectl Create the file ~/.kube/config (or replace it if it already exists).
Paste the contents of the k3s.yaml output into the config file.
Test that kubectl works by running the command
kubectl describe services
-
- Deploy kubectl kubectl apply -f deployment/db-configmap.yaml - Set up environment variables for the pods kubectl apply -f deployment/db-secret.yaml - Set up secrets for the pods kubectl apply -f deployment/postgres.yaml - Set up a Postgres database running PostGIS kubectl apply -f deployment/udaconnect-api.yaml - Set up the service and deployment for the API kubectl apply -f deployment/udaconnect-app.yaml - Set up the service and deployment for the web app sh scripts/run_db_command.sh <POD_NAME> - Seed your database against the postgres pod. (kubectl get pods will give you the POD_NAME)
'kubectl apply -f deployment/'
-
- Seed the Database The first time you run this project, you will need to seed the database with dummy data. Use the command:
sh scripts/run_db_command.sh <POD_NAME> against the postgres pod. (kubectl get pods will give you the POD_NAME). Subsequent runs of kubectl apply to make changes to deployments or services will not require you to seed the database again!
-
- Verify Your Deployment Once the project is up and running, you should be able to see three deployments and three services in Kubernetes. To verify, run the following commands:
kubectl get pods kubectl get services Both should both return udaconnect-app, udaconnect-api, and postgres.
http://localhost:30001/ - OpenAPI Documentation
http://localhost:30001/api/ - Base path for API
http://localhost:30000/ - Frontend ReactJS Application
To start, fork or clone the starter code.
The project is organized with the following structure:
db - database scripts deployment - Kubernetes configurations modules - application logic scripts - miscellaneous scripts When you're working on your services, it will make the most sense to add them under the modules directory with a similar setup to that of the starter code.
The project contains a README in the project root. This is a great resource for referencing any project instructions or details for the project.
- Review and Plan Review the starter project Determine which message passing strategies would integrate well when refactoring to a microservice architecture.
- Design and Document Using the design decisions from the previous step, create an architecture diagram of your microservice architecture showing the services and message passing techniques between them. Continue to use Kubernetes and maintain the core functionality of the starter project. Include at least three message passing strategies into your microservice architecture implementing Kafka, gRPC, and either enhancing or creating a RESTful API endpoint.
- Justify Your Decisions Write a 2-3 sentence rationale for each message passing strategy to justify your decision.
- Refactor into Microservices Refactor the starter code into a microservice architecture. While microservices can be technology-agnostic, we want to make sure that we use tools that your company is comfortable with. Therefore, this project should be done in Python.
- Create OpenAPI Documentation Provide OpenAPI documentation for API endpoints.
- Create a Postman Library Provide Postman library for REST endpoints that you created or modified.
The completed project must include:
- Instructions and commands on how to run the project in the project README.
- Architecture diagram named docs/architecture_design.png
- Document on justifying your architecture’s design decisions named docs/architecture_decisions.txt
- OpenAPI documentation of your new REST API endpoint as docs/openapi.yaml
- gRPC documentation of your endpoint and how to make a sample request in docs/grpc.txt
- Screenshot of kubectl get pods as docs/pods_screenshot.png
- Screenshot of kubectl get services as docs/services_screenshot.png
- All project code
- Postman collection of REST API endpoints that you created or modified as docs/postman.json
pip install grpcio-tools
python -m grpc_tools.protoc -I./ --python_out=./ --grpc_python_out=./ order.proto
##Summary_1 Protobuf File proto3 is specified as the syntax OrderMessage uses enums Status and Equipment to add string validation Empty message is defined to help handle null values where an empty argument is passed to the service OrderMessageList helps capture multiple OrderMessage's OrderService defines Create and Get stubs OrderService.Get has no argument so it uses the Empty message as its input
##Summary_2 Server Code Server code is defined in main.py using mostly boilerplate code to set up a gRPC server OrderServicer.Get returns hardcoded data by creating two order_pb2.OrderMessage objects and returning them in the OrderMessageList OrderServicer.Create returns the same data that was passed in Running the gRPC Server python main.py will run the gRPC server and print its outputs Create gRPC Client Writer Writer is used to demonstrate how OrderServicer.Create is called Defined in writer.py Creates an OrderMessage and passes it to OrderServicer.Create Getter Getter is used to demonstrate how OrderServicer.Get is called Defined in getter.py Creates an Empty message and passes it to OrderServicer.Get Running gRPC Clients python getter.py returns a hard-coded OrderMessageList python writer.py returns the data that we passed in gRPC server behavior can be validated as it prints the output