Azure Terraform Quickstart template

Use this template to easily create a new Git Repository for managing Jenkins X cloud infrastructure needs.

We recommend using Terraform to manange the infrastructure needed to run Jenkins X. There are a number of cloud resources which may need to be created such as:

Kubernetes cluster
Storage buckets for long term storage of logs
IAM Bindings to manage permissions for applications using cloud resources

Jenkins X likes to use GitOps to manage the lifecycle of both infrastructure and cluster resources. This requires two Git Repositories to achieve this:

Infrastructure git repository: infrastructure resources will be managed by Terraform and will keep resources in sync.
Cluster git repository: the Kubernetes specific cluster resources will be managed by Jenkins X and keep resources in sync.

Prerequisites

A Git organisation that will be used to create the GitOps repositories used for Jenkins X below. e.g. https://github.com/organizations/plan.
Create a git bot user (different from your own personal user) e.g. https://github.com/join and generate a personal access token, this will be used by Jenkins X to interact with git repositories. e.g. https://github.com/settings/tokens/new?scopes=repo,read:user,read:org,user:email,write:repo_hook,delete_repo,admin:repo_hook
This bot user needs to have write permission to write to any git repository used by Jenkins X. This can be done by adding the bot user to the git organisation level or individual repositories as a collaborator Add the new bot user to your Git Organisation, for now give it Owner permissions, we will reduce this to member permissions soon.
Install terraform CLI - see here
Install jx CLI - see here
An Azure account, with the required tools as described in this guide

Git repositories

We use 2 git repositories:

Infrastructure git repository for the Terraform configuration to setup/upgrade/modify your cloud infrastructure (kubernetes cluster, IAM accounts, IAM roles, buckets etc)
Cluster git repository to contain the helmfile.yaml file to define the helm charts to deploy in your cluster

We use separate git repositories since the infrastructure tends to change rarely; whereas the cluster git repository changes a lot (every time you add a new quickstart, import a project, release a project etc).

Often different teams look after infrastructure; or you may use tools like Terraform Cloud to process changes to infrastructure & review changes to infrastructure more closely than promotion of applications.

Getting started

Note: remember to create the Git repositories below in your Git Organisation rather than your personal Git account else this will lead to issues with ChatOps and automated registering of webhooks.

Create and clone your Infrastructure git repo from this GitHub Template https://github.com/jx3-gitops-repositories/jx3-terraform-azure/generate

Note: Ensure Owner is the name of the Git Organisation that will hold the GitOps repositories used for Jenkins X.
Create a Cluster git repository; choosing your desired secrets store, either Vault or Azure Key Vault:
- Vault: https://github.com/jx3-gitops-repositories/jx3-azure-vault/generate
- Azure Key Vault: https://github.com/jx3-gitops-repositories/jx3-azure-akv/generate
Note: Ensure Owner is the name of the Git Organisation that will hold the GitOps repositories used for Jenkins X.
You need to configure the git URL of your Cluster git repository (which contains helmfile.yaml) into the Infrastructure git repository (which contains main.tf).

So from inside a git clone of the Infrastructure git repository (which already has the file main.tf inside) you need to link to the other Cluster repository (which contains helmfile.yaml) by committing the required terraform values from below.

The contents of your values.auto.tfvars file should look something like this ....

jx_git_url = "https://github.com/myowner/myname-cluster"
jx_bot_username = "bot_user"

commit and push any changes to your Infrastructure git repository:

git commit -a -m "fix: configure cluster repository and project"
git push

Now define an environment variable to pass the token into Terraform:

export TF_VAR_jx_bot_token=my-bot-token

Now, initialise, plan and apply Terraform:

terraform init

terraform plan

terraform apply

Connect to the cluster

$(terraform output connect)

Tail the Jenkins X installation logs

$(terraform output follow_install_logs)

Once finished you can now move into the Jenkins X Developer namespace

jx ns jx

and create or import your applications

jx project

Terraform Inputs

Name	Description	Type	Default	Required
apex_domain	The name of the parent/apex domain in which to create this domain zone, e.g. jenkins-x.io. Required if dns_enabled set to true	`string`	`""`	no
apex_domain_integration_enabled	Flag that when set attempts to create delegation records in apex domain to point to domain created by this module	`bool`	`false`	no
apex_resource_group_name	The resource group in which the Azure DNS apex domain resides. Required if apex_domain_integration_enabled is true	`string`	`""`	no
build_node_count	The number of build nodes to use for the cluster	`number`	`null`	no
build_node_size	The size of the build node to use for the cluster	`string`	`""`	no
cluster_name	Variable to provide your desired name for the cluster. The script will create a random name if this is empty	`string`	`""`	no
cluster_network_model	Variable to define the network model for the cluster. Valid values are either `kubenet` or `azure`	`string`	`"kubenet"`	no
cluster_node_resource_group_name	Resource group name in which to provision AKS cluster nodes. The script will create a random name if this is empty	`string`	`""`	no
cluster_resource_group_name	The name of the resource group in to which to provision AKS managed cluster. The script will create a random name if this is empty	`string`	`""`	no
cluster_version	Kubernetes version to use for the AKS cluster	`string`	`"1.20.7"`	no
dns_prefix	DNS prefix for the cluster. The script will create a random name if this is empty	`string`	`""`	no
dns_resource_group_name	Resource group in which to create the Azure DNS zone. The script will create a random name if this is empty	`string`	`""`	no
enable_log_analytics	Flag to indicate whether to enable Log Analytics integration for cluster	`bool`	`false`	no
jx_bot_token	Bot token used to interact with the Jenkins X cluster git repository	`string`	n/a	yes
jx_bot_username	Bot username used to interact with the Jenkins X cluster git repository	`string`	n/a	yes
jx_git_url	URL for the Jenkins X cluster git repository	`string`	n/a	yes
key_vault_enabled	Flag to indicate whether to provision Azure Key Vault for secret storage	`string`	`true`	no
key_vault_name	Name of Azure Key Vault to create	`string`	`""`	no
key_vault_resource_group_name	Resource group to create in which to place key vault	`string`	`""`	no
key_vault_sku	SKU of the Key Vault resource to create. Valid values are standard or premium	`string`	`"standard"`	no
location	The Azure region in to which to provision the cluster	`string`	`"australiaeast"`	no
logging_retention_days	Number of days to retain logs in Log Analytics if enabled	`number`	`30`	no
max_build_node_count	The maximum number of builder nodes to use for the cluster if autoscaling is enabled	`number`	`null`	no
max_node_count	The maximum number of worker nodes to use for the cluster if autoscaling is enabled	`number`	`null`	no
min_build_node_count	The minimum number of builder nodes to use for the cluster if autoscaling is enabled	`number`	`null`	no
min_node_count	The minimum number of worker nodes to use for the cluster if autoscaling is enabled	`number`	`null`	no
network_name	The name of the Virtual Network in Azure to be created. The script will create a random name if this is empty	`string`	`""`	no
network_resource_group_name	The name of the resource group in to which to provision network resources. The script will create a random name if this is empty	`string`	`""`	no
node_count	The number of worker nodes to use for the cluster	`number`	`2`	no
node_size	The size of the worker node to use for the cluster	`string`	`"Standard_B2ms"`	no
registry_resource_group_name	The name of the resource group in to which to provision ACR managed registry. The script will create a random name if this is empty	`string`	`""`	no
server_side_apply_enabled	BETA: Flag to indicate to the jx-git-operator that you would like to use server side apply	`bool`	`false`	no
spot_max_price	The maximum price you're willing to pay in USD per virtual machine, -1 to go to the maximum price	`number`	`-1`	no
storage_resource_group_name	Resource group to create in which to place storage accounts	`string`	`""`	no
subdomain	The subdomain name of the zone to create, e.g. dev-subdomain.	`string`	`""`	no
subnet_cidr	The CIDR of the provisioned subnet within the `vnet_cidr` to to which worker nodes are placed	`string`	`"10.8.0.0/24"`	no
subnet_name	The name of the subnet in Azure to be created. The script will create a random name if this is empty	`string`	`""`	no
use_existing_acr_name	Name of the existing ACR that you would like to use, e.g. use this in multicluster setup, when you want to use DEV cluster ACR.	`string`	`null`	no
use_existing_acr_resource_group_name	Name of the resources group of the existing ACR that you would like to use, e.g. use this in multicluster setup, when you want to use DEV cluster ACR.	`string`	`null`	no
use_spot	Should we use spot instances for the build nodes	`bool`	`true`	no
vnet_cidr	The CIDR of the provisioned Virtual Network in Azure in to which worker nodes are placed	`string`	`"10.8.0.0/16"`	no

Cleanup

To remove any cloud resources created here run:

terraform destroy

Contributing

When adding new variables please regenerate the markdown table

terraform-docs markdown table .

and replace the Inputs section above

Formatting

When developing please remember to format codebase before raising a pull request

terraform fmt -check -diff -recursive

okutasan / test