Azure Kubernetes service makes it easy to spin up a functional cluster, but there is more to running Kubernetes than the nodes. Making a cluster workable requires good amount of grunt work.
This is an attempt to make it easier to adopt AKS for modern microservice based cloud native applications.
Microservice apps need supporting components to do their job. Selecting what components to use can be daunting. In this repo we have selected components based on experience from projects we have worked on, that we have found most suitable and that make full use of Azure as a cloud computing platform.
The driving force behind this repository is to help you use Azure to its fullest and to let you focus less on configuration and component choices, and more on writing business logic.
This project derives the concept for functional AKS cluster from Project Bedrock. Bedrock provides patterns, implementation, and automation for operating production Kubernetes clusters based on a GitOps workflow, building on the best practices discovered while working with operationalizing Kubernetes clusters.
MagicAKS extends the concepts in Bedrock to initialize a cluster with opinionated choice of technologies that work well together. This level on integration provides for a smoother development experience with tooling, monitoring and other infrastructure services. While Bedrock provides guidance for Azure DevOps based release workflows, MagicAKS focuses on Github repositories and actions. Both projects use Fabrikate for specifying cluster software configs but Fabrikate is a replaceable component and one can use for example Kustomize or any other manifest generation technology.
Bedrock is designed to be general and has some concepts like rings are not reflected in this project at all. MagicAKS focuses specifically on making microservices based development on Azure easy.
This repository is by design not general purpose. It simplifies deployment by removing thinking of choices and instead get a working production grade kubernetes cluster so you as a developer can write business logic instead of spending time to figure out infrastructure concerns.
The architecture includes the following components:
- Azure Service Bus as async message bus
- Azure Container Insights / Azure Monitor for logging and metrics store
- Azure Active Directory for authentication and authorization. Integrated with K8s RBAC
- Azure Key Vault for secret storage
- Azure Policy for policy enforcement, compliance and governance (Done)
Planned components
- Azure SQL as database provider.
- Azure Application Gateway as ingress controller.
When needed, the solution uses open source tooling like kured to keep cluster nodes up to date.
This solution consists of 3 stages.
- Pre-provision - Provisioning base resources that can be reused for multiple AKS clusters
- Provision AKS - Provisioning the AKS cluster
- Post-provision - Provisioning supporting resources and tools
We use terraform to bootstrap the infrastructure and set up a gitOps connection in the cluster which then picks up kubernetes manifests for the declared state of the cluster.
These manifests are in turn divided into two repositories.
- One repository with manifests which require admin permissions on the cluster to install.
- One repository with manifests for workloads which can run with limited credentials.
This is done to prevent unauthorized installations in the cluster. Hence, there are two flux pods running, one for each type of manifest. Admin manifests should come from a repository controlled by cluster admins that has tighter process control methods that checks every input before it's applied to the cluster.
Instead of writing k8s manifests directly we use Fabrikate High Level Definition (HLD) to write the config which is then translated to K8s manifests using a pipeline which in turn runs fab generate and pushes the config to the k8s manifest repo.
If you use a different manifest generation system make sure you to run rbac-generator.py and azmonconfig-generator.py as part of building the manifests.
Set up your environment using the setup instructions
Check open issues at Github Issues