This repository contains the tool and the policy library for validating GKE clusters against configuration best practices and scalability limits.
Note: this is not an officially supported Google product.
The container images with GKE Policy Automation tool are hosted on ghcr.io. Check the packages page
for a list of all tags and versions.
docker pull ghcr.io/google/gke-policy-automation:latest
docker run --rm ghcr.io/google/gke-policy-automation check \
-project my-project -location europe-west2 -name my-clusterThe GKE Policy Automation is available as a Krew plugin.
kubectl krew install gke-policy
kubectl gke-policy check --discovery -p my-projectBinaries for Linux, Windows and Mac are available as tarballs in the release page.
Go v1.21 or newer is required. Check the development guide for more details.
git clone https://github.com/google/gke-policy-automation.git
cd gke-policy-automation
make build
./gke-policy check \
--project my-project --location europe-west2 --name my-clusterFull user guide: GKE Policy Automation User Guide.
The configuration best practices check validates GKE clusters against the set of GKE configuration policies.
./gke-policy check \
--project my-project --location europe-west2 --name my-clusterThe scalability limits check validates GKE clusters against the GKE quotas and limits. The tool will report violations when the current values will cross the certain thresholds.
./gke-policy check scalability \
--project my-project --location europe-west2 --name my-clusterNOTE: you need to run kube-state-metrics to export cluster metrics to use cluster scalability
limits check. Refer to the kube-state-metrics installation & configuration guide
for more details.
The tool assumes that metrics are available in Cloud Monitoring, i.e. in a result of Google Cloud Managed Service for Prometheus based metrics collection. If self managed Prometheus collection is used, be sure to:
-
Configure Prometheus scraping for
kube-state-metricsusingPodMonitor/ServiceMonitorand corresponding annotations, i.e.prometheus.io/scrape -
Configure custom Prometheus API server address in a tool
-
Prepare
config.yaml:inputs: metricsAPI: enabled: true address: http://my-prometheus-svc:8080 # Prometheus server API endpoint username: user # username for basic authentication (optional) password: secret # password for basic authentication (optional)
-
Run
./gke-policy check scalability -c config.yaml
-
The common options apply to all types of check commands.
Check multiple GKE clusters using the config file.
./gke-policy check -c config.yamlThe config.yaml file:
clusters:
- name: prod-central
project: my-project-one
location: europe-central2
- id: projects/my-project-two/locations/europe-west2/clusters/prod-westCheck multiple clusters by discovering them in a selected GCP projects, folders or in the entire organization using Cloud Asset Inventory and configuration file.
./gke-policy check -c config.yamlThe config.yaml file:
clusterDiscovery:
enabled: true
organization: "123456789012"It is possible to use cluster discovery on a given project using command line flags only:
./gke-policy check --discovery -p my-project-idData for cluster validation can be retrieved from multiple data sources, eg. GKE API, Cloud Monitoring API or local JSON file exported from GKE API. For best practices checks GKE API is enabled by default, and for scalability checks, metrics API is enabled as well. Check Inputs user guide for more details.
Example:
- Metrics API input from Cloud Monitoring configured in dedicated project and other values set with defaults for scalability check
inputs:
gkeAPI:
enabled: true
gkeLocal:
enabled: false
file:
metricsAPI:
enabled: true
project: sample-project
metrics:The cluster validation results can be published to multiple outputs, including JSON file, Pub/Sub topic, Cloud Storage bucket or Security Command Center. Check Outputs user guide for more details.
Examples:
-
JSON file output with command line flags
./gke-policy check \ --project my-project --location europe-west2 --name my-cluster \ --out-file output.json
-
All outputs enabled in a configuration file
clusters: - name: my-cluster project: my-project location: europe-west2 outputs: - file: output.json - pubsub: topic: Test project: my-pubsub-project - cloudStorage: bucket: bucket-name path: path/to/write - securityCommandCenter: organization: "153963171798"
Specify custom repository with the GKE cluster best practices and check the cluster against them.
-
Custom policies source with command line flags
./gke-policy check \ --project my-project --location europe-west2 --name my-cluster \ --git-policy-repo "https://github.com/google/gke-policy-automation" \ --git-policy-branch "main" \ --git-policy-dir "gke-policies-v2"
-
Custom policies source with configuration file
./gke-policy check -c config.yaml
The
config.yamlfile:clusters: - name: my-cluster project: my-project location: europe-west2 policies: - repository: https://domain.com/your/custom/repository branch: main directory: gke-policies-v2
The tool is fetching GKE cluster details using GCP APIs. The application default credentials are used by default.
- When running the tool in GCP environment, the tool will use the attached service account by default
- When running locally, use
gcloud auth application-default logincommand to get application default credentials - To use credentials from service account key file pass
--credsparameter with a path to the file.
The minimum required IAM role is roles/container.clusterViewer
on a cluster projects. Additional roles may be needed, depending on configured outputs
- check authentication section in the user guide.
The GKE Policy Automation tool can be executed in a serverless way to perform automatic evaluations of a clusters running in your organization. Please check our reference Terraform Solution that leverages GCP serverless solutions including Cloud Scheduler and Cloud Run.
Please check out Contributing and Code of Conduct docs before contributing.
Please check GKE Policy Automation development for guides on building and developing the application.
Please check GKE Policy authoring guide for guides on authoring REGO rules for GKE Policy Automation.