terraform-google-composer
This module makes it easy to create a Cloud Composer Environment. As the module develops, this README should be updated.
The resources/services/activations/deletions that this module will create/trigger are:
- Create a GCP Composer Environment
Usage
Cloud Composer currently has two versions: V2 has greater flexibility in the Airflow core services (scheduler, webserver, worker) and has a more practical and scalable infrastructure. Therefore, we recommend prioritizing the use of V2 for new environments.
You can find an overview of the product here and the list of major differences.
Simple usage is as follows:
module "composer" {
source = "terraform-google-modules/composer/google"
version = "~> 3.4"
project_id = "<PROJECT ID>"
region = "us-central1"
composer_env_name = "composer-env-test"
network = "test-network"
subnetwork = "composer-subnet"
enable_private_endpoint = false
}Functional examples are included in the examples directory.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| composer_env_name | Name of Cloud Composer Environment | string |
n/a | yes |
| enable_private_endpoint | Configure public access to the cluster endpoint. | bool |
false |
no |
| network | Network where Cloud Composer is created. | string |
n/a | yes |
| project_id | Project ID where Cloud Composer Environment is created. | string |
n/a | yes |
| region | Region where the Cloud Composer Environment is created. | string |
n/a | yes |
| subnetwork | Subetwork where Cloud Composer is created. | string |
n/a | yes |
| zone | Zone where the Cloud Composer Environment is created. | string |
n/a | yes |
Outputs
| Name | Description |
|---|---|
| airflow_uri | URI of the Apache Airflow Web UI hosted within the Cloud Composer Environment. |
| composer_env_id | ID of Cloud Composer Environment. |
| composer_env_name | The name of the Cloud Composer Environment. |
| gcs_bucket | Google Cloud Storage bucket which hosts DAGs for the Cloud Composer Environment. |
| gke_cluster | Google Kubernetes Engine cluster used to run the Cloud Composer Environment. |
Requirements
These sections describe requirements for using this module.
Software
The following dependencies must be available:
- Terraform v0.13+
- Terraform Provider for GCP plugin v3.53+
Service Account
A service account with the following roles must be used to provision the resources of this module:
- Project Editor:
roles/editor - Network Admin:
roles/compute.networkAdmin - Instance Admin:
roles/compute.instanceAdmin.v1 - Service Account User:
roles/iam.serviceAccountUser - Composer Worker:
roles/composer.worker
The Project Factory module and the IAM module may be used in combination to provision a service account with the necessary roles applied.
APIs
A project with the following APIs enabled must be used to host the resources of this module:
- Cloud Composer API:
composer.googleapis.com
The Project Factory module can be used to provision a project with the necessary APIs enabled.
Contributing
Refer to the contribution guidelines for information on contributing to this module.