Cloud-Storage-Pubsub
Developed a containerized service which publishes a data as a message to pub sub and then push the container image to GCR and deploy service using GKE
Test Enviornment
http://34.94.209.219/
Before you begin
- You're new to Google Cloud, create an account.
- The Google Cloud Console, on the project selector page, select or create a Google Cloud project (capstone1-project).
- Create a bucket named currency-bucket-egen.
- Enable the Artifact Registry, Cloud Build, Cloud Function and Google Kubernetes Engine APIs.
- Install and initialize cloud SDK
Create a PubSub topic and Cloud function to trigger the PubSub
- Go to the Pub/Sub topics page in the Cloud Console.
- Click Create a topic named currency.
- Create a subscription
- Leave the delivery type as Pull.
- Create a cloud function, name the cloud function as loadStorage using US-west2 region and pubsub as a trigger.
- upload main.py and entry point as currency and then deploy it.
Get the data from API
- install all the files required in requirements.txt
- GetDataToPub.py is responsible to get data from the API and publish it to the Pub and from there cloud function will sense the event and trigger the subscription. and the data will get to the bucket.
Push container image to GCR
Before you begin:
On the cloudshell- gcloud components install kubectl create a directory on cloud shell editor (egen-capstone) cd into the directory you just created.
Routing function
app.py has a restful api which works with the http and whenever that link is hit the GetDataToPub.py is triggred and then the data is stored to the Google cloud bucket.
Containerizing an app with Cloud Build
- To containerize the sample app, create a new file named Dockerfile in the same directory as the source files.
- Add a .dockerignore file to ensure that local files don't affect the container build process (hidden file).
- you will get the Dockerfile from this repository.
Store your container in Artifact Registry
gcloud artifacts repositories create capstone-repo \
--project=capstone1-project-326220 \
--repository-format=docker \
--location=us-west2 \
--description="Docker repository for captone project"
Build your container image using Cloud Build
gcloud builds submit \
--tag us-west2-docker.pkg.dev/capstone1-project-326220/capstone-repo/egen-capstone
Now the image is stored in Artifact Registry.
Creating a GKE cluster
gcloud container clusters create egen-cluster \
--zone us-west2
Connect to GKE cluster
Verify that you have access to the cluster.
kubectl get nodes
Deploying to GKE
To deploy your app to the GKE cluster you created, you need two Kubernetes objects.
A Deployment - to define your app. A Service - to define how to access your app.
Deploy an app
- Copy the deployment.yaml file in the same directory as your other files.
- Deploy the resource to the cluster:
kubectl apply -f deployment.yaml
- Track the deployment:
kubectl get deployments
Deploy a Service
- Copy the file service.yaml in the same directory as your other source files.
- Create the Hello World Service:
kubectl apply -f service.yaml
- Get the external IP address of the Service:
kubectl get services
- View a deployed app: http://<external_ip>