terraform-ibm-modules / terraform-ibm-observability-agents

Deploys the Log Analysis agent and Cloud Monitoring agents to a cluster

Terraform IBM Observability agents module

This module deploys the following observability agents to a Red Hat OpenShift Container Platform cluster:

• Logging agent
• Monitoring agent

Overview

• terraform-ibm-observability-agents
• Examples
  • Deploy basic observability agents
• Contributing

terraform-ibm-observability-agents

Usage

# ############################################################################
# Init cluster config for helm
# ############################################################################

data "ibm_container_cluster_config" "cluster_config" {
  # update this value with the Id of the cluster where these agents will be provisioned
  cluster_name_id = "cluster_id"
}

# ############################################################################
# Config providers
# ############################################################################

provider "ibm" {
  # update this value with your IBM Cloud API key value
  ibmcloud_api_key = "api key value"  # pragma: allowlist secret
}

provider "helm" {
  kubernetes {
    host                   = data.ibm_container_cluster_config.cluster_config.host
    token                  = data.ibm_container_cluster_config.cluster_config.token
    cluster_ca_certificate = data.ibm_container_cluster_config.cluster_config.ca_certificate
  }
}

# ############################################################################
# Install observability agents
# ############################################################################

module "observability_agents" {
  source                           = "terraform-ibm-modules/observability-agents/ibm"
  version                          = "X.X.X" # Replace "X.X.X" with a release version to lock into a specific release
  cluster_id                       = "cluster id" # update this with your cluster id where the agents will be installed
  cluster_resource_group_id        = "resource group id" # update this with the Id of your IBM Cloud resource group
  log_analysis_ingestion_key       = "XXXXXXXX"
  log_analysis_instance_region     = "us-south"
  cloud_monitoring_access_key      = "XXXXXXXX"
  cloud_monitoring_instance_region = "us-south"
}

Configuration for Kubernetes metadata filtering in the logging agent

You can configure the logging agent to filter log lines according to the Kubernetes resources metadata by setting the exclusion and inclusion parameters.

For example, to set the agent to return all log lines coming from the default Kubernetes namespace and exclude anything with a label app.kubernetes.io/name with value sample-app or an annotation annotation.user with value sample-user, include these parameters:

custom_log_analysis_at_agent_line_exclusion = "label.app.kubernetes.io/name:sample-app\\, annotation.user:sample-user"
custom_log_analysis_at_agent_line_inclusion = "namespace:default"

The following is the corresponding DaemonSet configuration:

- name: LOGDNA_K8S_METADATA_LINE_INCLUSION
  value: "label.app.kubernetes.io/name:sample-app, annotation.user:sample-user"
- name: LOGDNA_K8S_METADATA_LINE_EXCLUSION
  value: "namespace:default"

For more information, see Configuration for Kubernetes Metadata Filtering.

Required IAM access policies

You need the following permissions to run this module.

• IAM Services
  • IBM Cloud Activity Tracker service
    • Viewer platform access
    • Reader service access
  • IBM Cloud Monitoring service
    • Viewer platform access
    • Reader service access
  • IBM Log Analysis service
    • Viewer platform access
    • Reader service access
  • Kubernetes service
    • Viewer platform access
    • Manager service access

Requirements

Name	Version
terraform	>= 1.1.0, <1.6.0
helm	>= 2.8.0, <3.0.0
ibm	>= 1.59.0, <2.0.0

Modules

No modules.

Resources

Name	Type
helm_release.cloud_monitoring_agent	resource
helm_release.log_analysis_agent	resource
ibm_container_cluster_config.cluster_config	data source
ibm_container_vpc_cluster.cluster	data source

Inputs

Name	Description	Type	Default	Required
cloud_monitoring_access_key	Access key used by the IBM Cloud Monitoring agent to communicate with the instance	string	null	no
cloud_monitoring_add_cluster_name	If true, configure the cloud monitoring agent to attach a tag containing the cluster name to all metric data.	bool	true	no
cloud_monitoring_agent_name	Cloud Monitoring agent name. Used for naming all kubernetes and helm resources on the cluster.	string	"sysdig-agent"	no
cloud_monitoring_agent_namespace	Namespace where to deploy the Cloud Monitoring agent. Default value is 'ibm-observe'	string	"ibm-observe"	no
cloud_monitoring_agent_tags	List of tags to associate to all matrics that the agent collects. NOTE: Use the 'cloud_monitoring_add_cluster_name' variable to add the cluster name as a tag.	list(string)	[]	no
cloud_monitoring_agent_tolerations	List of tolerations to apply to Cloud Monitoring agent.	list(object({ key = optional(string) operator = optional(string) value = optional(string) effect = optional(string) tolerationSeconds = optional(number) }))	[ { "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master", "operator": "Exists" } ]	no
cloud_monitoring_enabled	Deploy IBM Cloud Monitoring agent	bool	true	no
cloud_monitoring_endpoint_type	Specify the IBM Cloud Monitoring instance endpoint type (public or private) to use. Used to construct the ingestion endpoint.	string	"private"	no
cloud_monitoring_instance_region	The IBM Cloud Monitoring instance region. Used to construct the ingestion endpoint.	string	null	no
cloud_monitoring_metrics_filter	To filter custom metrics, specify the Cloud Monitoring metrics to include or to exclude. See https://cloud.ibm.com/docs/monitoring?topic=monitoring-change_kube_agent#change_kube_agent_inc_exc_metrics.	list(object({ type = string name = string }))	[]	no
cloud_monitoring_secret_name	The name of the secret which will store the access key.	string	"sysdig-agent"	no
cluster_config_endpoint_type	Specify which type of endpoint to use for for cluster config access: 'default', 'private', 'vpe', 'link'. 'default' value will use the default endpoint of the cluster.	string	"default"	no
cluster_id	The ID of the cluster you wish to deploy the agents in	string	n/a	yes
cluster_resource_group_id	The Resource Group ID of the cluster	string	n/a	yes
log_analysis_add_cluster_name	If true, configure the log analysis agent to attach a tag containing the cluster name to all log messages.	bool	true	no
log_analysis_agent_custom_line_exclusion	Log Analysis agent custom configuration for line exclusion setting LOGDNA_K8S_METADATA_LINE_EXCLUSION. See https://github.com/logdna/logdna-agent-v2/blob/master/docs/KUBERNETES.md#configuration-for-kubernetes-metadata-filtering for more info.	string	null	no
log_analysis_agent_custom_line_inclusion	Log Analysis agent custom configuration for line inclusion setting LOGDNA_K8S_METADATA_LINE_INCLUSION. See https://github.com/logdna/logdna-agent-v2/blob/master/docs/KUBERNETES.md#configuration-for-kubernetes-metadata-filtering for more info.	string	null	no
log_analysis_agent_name	Log Analysis agent name. Used for naming all kubernetes and helm resources on the cluster.	string	"logdna-agent"	no
log_analysis_agent_namespace	Namespace where to deploy the Log Analysis agent. Default value is 'ibm-observe'	string	"ibm-observe"	no
log_analysis_agent_tags	List of tags to associate to all log records that the agent collects so that you can identify the agent's data quicker in the logging UI. NOTE: Use the 'log_analysis_add_cluster_name' variable to add the cluster name as a tag.	list(string)	[]	no
log_analysis_agent_tolerations	List of tolerations to apply to Log Analysis agent.	list(object({ key = optional(string) operator = optional(string) value = optional(string) effect = optional(string) tolerationSeconds = optional(number) }))	[ { "operator": "Exists" } ]	no
log_analysis_enabled	Deploy IBM Cloud Logging agent	bool	true	no
log_analysis_endpoint_type	Specify the IBM Log Analysis instance endpoint type (public or private) to use. Used to construct the ingestion endpoint.	string	"private"	no
log_analysis_ingestion_key	Ingestion key for the IBM Cloud Logging agent to communicate with the instance	string	null	no
log_analysis_instance_region	The IBM Log Analysis instance region. Used to construct the ingestion endpoint.	string	null	no
log_analysis_secret_name	The name of the secret which will store the ingestion key.	string	"logdna-agent"	no

Outputs

No outputs.

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.