github-actions github-packages golang kubernetes

Overview

ANZ technical zeyu

1. Pre-requirements (to be setup locally)

1.1 Git

assuming you are familiar with git (and already have git installed and configured

git clone https://github.com/Shuliyey/technical-tests.git
cd technical-tests

1.2 Local build environment requirements

below terminal applications/tools needs to be installed

bash
docker
make
curl
git
go
terraform (>=1.3.0, https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)
kubernetes-cli (kubectl, https://kubernetes.io/docs/tasks/tools/)
gcloud (e.g debian distro sudo apt-get install google-cloud-sdk, https://cloud.google.com/sdk/docs/install)
gke-gcloud-auth-plugin (e.g debian distro sudo apt-get install google-cloud-sdk-gke-gcloud-auth-plugin, https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke)

2. Tasks/Steps

2.1 Q1 - Dockerfile Multistage Build and Optmisation

The finalised Dockerfile is available at Dockerfile

multi-stage build
- the builder section is responsible for producing the final application binary /app/golang-test
- the final docker image uses alpine as the minimal based image and port the final application /app/golang-test produced by the builder section. This keeps the final image at its minimum, as the runtime image doesn't need the full dependency for building the application (golang packages dependent libraries along side with the application into a single executable binary)
cache optmisation
- Dockerfile structure has been designed to achieve cache optmisation as it puts the static defined steps before the dynamic dependend steps, meaning the static steps will be cached for optmisation

2.2 Q2 - Application, Containerisation and CICD pipeline

for more details on

cicd - branching strategy, versioning, release page, github actions and etc.
testing - unit testing, lint test
running/building the application locally
dockerization on the application

refer to q2-application-containerisation-and-cicd-pipeline

2.2.1 make

generate info - generates the application related information (e.g version, lastcommitsha, description and etc)

make info.generate

test - runs lint and unit test

make go.test

run locally - runs application on local workstation

make go.run

build locally - builds application on local workstation

make go.build

build docker image - builds docker image locally with certain convention (e.g uses default short git commit sha as <tag>), behaviour can be altered based on environment variables.

make docker.build

run docker image - runs docker image locally with certain convention (e.g uses binds to port 8000 on host workstation), behaviour can be altered based on environment variables.

make docker.run

Note: make sure the HOST_PORT is available and not in use by other processes

there'll be message indicating the local url to visit (running in local mode (docker run), visit app at localhost:HOST_PORT/version), below is an example

2.2.2 github-actions

cicd pipeline is implemented in github actions, pipeline is defined in .github/workflows/go.yml

quick overview of github-actions

on: defines set of github events that will trigger the pipeline

on:
  push:
    branches:
      - '*/*'
      - master
    tags:
      - v*
  pull_request:
    branches:
      - master

In our case above, the pipeline will trigger on push event matching branch */* or master and matchings tags v*, pull_request event against destination branch master

jobs: defines a set of jobs that will get triggered.

jobs:
  golang:
    name: Golang Pipline
    runs-on: ubuntu-latest

In our case above golang job will be triggered and it runs on ubuntu-latest vm

steps: defines a sequence of steps to be run in order

      - name: Publish to Github Registry
        id: publish-github-registry
        uses: elgohr/Publish-Docker-Github-Action@v5
        env:
          GO_VERSION: 1.20.2
          ALPINE_VERSION: 3.17
        with:
          name: ${{ steps.calculate-docker-info.outputs.repository }}/app2
          registry: docker.pkg.github.com
          username: ${{ steps.calculate-docker-info.outputs.actor }}
          password: ${{ github.token }}
          buildargs: GO_VERSION,ALPINE_VERSION
          tags: ${{ steps.calculate-docker-info.outputs.tags }}
          cache: ${{ github.event_name != 'schedule' }}
        if: contains(github.ref, 'refs/tags/v') || contains(github.ref, 'refs/heads/master')

In our case above, the step have id publish-github-registry and reuses elgohr/Publish-Docker-Github-Action@v5 github-action template (this is a github repo available at elgohr/Publish-Docker-Github-Action), with: condition defines set of parameters to be passed to the github-action template, lastly if: condition defines the criteria that must match (or else the step will be skipped)

list of triggered pipelines can be visited on the repo's github-actions page (https://github.com/Shuliyey/technical-tests/actions)

below are three examples of triggered pipelines

456994692 (on push to master branch)
456989568 (on push to '*/*' branch)
456992006 (on pull_request to master branch)
456898515 (on push 'v*' tags)

#4 is an example pull requests (github-actions pipeline will runs a set of test to ensure quality of the pull request before it is merged to master (the "stable" branch), preventing bad code gets merged to "stable" master branch)

2.3 Deployments (Kubernetes)

2.3.1 make

assuming your local kubectl is already pointing to the desired kubernetes context (this can be verified through kubectl config current-context)

deploy application manifests to kubernetes

make k8s.deploy

there'll be a section (like below) indicating the endpoint to visit the application and more information such as tailing the application service logs

remove application manifests from kubernetes

make k8s.delete

more info refer to extra-deployments-kubernetes (note: the make k8s.deploy and make k8s.delete are self explanatory and does the creation and removal of the required application resources on kubernetes, this is just for more reference)

2.4 BONUS - GKE Infrastructure (terraform)

2.4.1 make

IMPORTANT NOTE: DO NOT RUN THIS IN WELL ESTABLISHED GCP PROJECT, RUN THIS IN DUMMY GCP PROJECT, TERRAFORM CODE INCLUDE ENABLEMENT OF GOOGLE APIS THROUGH google_project_service, WHEN TERRAFORM DESTROY IS RUN, IT COULD DELETE EXISTING GOOGLE APIS THAT'S ENABLED THROUGH OTHER METHODS

assuming your local gcloud/terraform is set up correctly (gcloud auth login, gcloud auth application-default login)

spin up GKE infrastructure

make terraform.apply

tear down GKE infrastructure

make terraform.destroy

more info refer to bonus-infrastructure-terraform (note: the make terraform.apply and make terraform.destroy are self explanatory and does the creation and removal of the required application resources on kubernetes, this is just for more reference)

2.5 ALL IN ONE

2.5.1 make

doing all of above with one command

spin up GKE infrastructure and deploy application to GKE

TF_VAR_project="<project>" make up

delete application from GKE and tear down GKE infrastructure

TF_VAR_project="<project>" make down

3. References

3.1 Github Actions & Github Packages

3.2 Kubernetes

3.3 terraform

About

Technical tests expected to be pre-interview tests to filter applicants.

github-actions github-packages golang kubernetes

Languages

Language:Shell 60.6%Language:Go 21.5%Language:HCL 12.5%Language:Makefile 3.4%Language:Dockerfile 2.0%