This document attempts to explain how the different components in Open Cluster Management Observabilty come together to deliver multicluster fleet observability. We do leverage several open source projects: Grafana, Alertmanager, Thanos, Observatorium Operator and API Gateway, Prometheus; We also leverage a few Open Cluster Mangement projects namely - Cluster Manager or Registration Operator, Klusterlet. The multicluster-observability operator is the root operator which pulls in all things needed.
Component | Git Repo | Description |
---|---|---|
MCO Operator | multicluster-observability-operator | Operator for monitoring. This is the root repo. If we follow the Readme instructions here to install, the code from all other repos mentioned below are used/referenced. |
Endpoint Operator | endpoint-metrics-operator | Operator that manages setting up observability and data collection at the managed clusters. |
Observatorium Operator | observatorium-operator | Operator to deploy the Observatorium project. Inside the open cluster management, at this time, it means metrics using Thanos. Forked from main observatorium-operator repo. |
Metrics collector | metrics-collector | Scrapes metrics from Prometheus at managed clusters, the metric collection being shaped by configuring allow-list. |
RBAC Proxy | rbac_query_proxy | Helper service that acts a multicluster metrics RBAC proxy. |
Grafana | grafana | Grafana repo - for dashboarding and metric analytics. Forked from main grafana repo. |
Dashboard Loader | grafana-dashboard-loader | Sidecar proxy to load grafana dashboards from configmaps. |
Management Ingress | management-ingress | NGINX based ingress controller to serve Open Cluster Management services. |
Observatorium API | observatorium | API Gateway which controls reading, writing of the Observability data to the backend infrastructure. Forked from main observatorium API repo. |
Thanos Ecosystem | kube-thanos | Kubernetes specific configuration for deploying Thanos. The observatorium operator leverages this configuration to deploy the backend Thanos components. |
- git
- go version v1.15+
- docker version 17.03+
- kubectl version v1.16.3+
- kustomize version v3.8.5+
- operator-sdk version v1.4.2+
- access to an OCP v4.6+ cluster
Note: By default, the API conversion webhook use on the Openshift service serving certificate feature to manage the certificate, you can replace it with cert-manager if you want to run the multicluster-observability-operator in a kubernetes cluster.
Use the following quick start commands for building and testing the multicluster-observability-operator:
Check out the multicluster-observability-operator repository.
$ git clone git@github.com:open-cluster-management/multicluster-observability-operator.git
$ cd multicluster-observability-operator
Build the multicluster-observability-operator image and push it to a public registry, such as quay.io:
$ make docker-build docker-push IMG=quay.io/<YOUR_USERNAME_IN_QUAY>/multicluster-observability-operator:latest
- Create the
open-cluster-management-observability
namespace if it doesn't exist:
$ kubectl create ns open-cluster-management-observability
- Deploy the minio service which acts as storage service of the multicluster observability:
$ kubectl -n open-cluster-management-observability apply -f examples/minio
- Replace the operator image and deploy the multicluster-observability-operator:
$ make deploy IMG=quay.io/<YOUR_USERNAME_IN_QUAY>/multicluster-observability-operator:latest
- Deploy the multicluster-observability-operator CR:
$ kubectl apply -f config/samples/observability_v1beta2_multiclusterobservability.yaml
- Verify all the components for the Multicluster Observability are starting up and runing:
$ kubectl -n open-cluster-management-observability get pod
NAME READY STATUS RESTARTS AGE
alertmanager-0 2/2 Running 0 5m
grafana-6878c8b44-kxx6k 2/2 Running 0 5m
minio-79c7ff488d-rqmfg 1/1 Running 0 5m
observability-observatorium-api-8646457ff9-6kw5v 1/1 Running 0 5m
observability-thanos-compact-0 1/1 Running 0 5m
observability-thanos-query-fdc9b77b-7hxgc 1/1 Running 0 5m
observability-thanos-query-frontend-8764896b99597 1/1 Running 0 5m
observability-thanos-receive-controller-86982stcb 1/1 Running 0 5m
observability-thanos-receive-default-0 1/1 Running 0 5m
observability-thanos-rule-0 2/2 Running 0 5m
observability-thanos-store-memcached-0 2/2 Running 0 5m
observability-thanos-store-shard-0-0 1/1 Running 0 5m
observatorium-operator-845dc69ccf-gdzn2 1/1 Running 0 5m
rbac-query-proxy-559b788777-ssmls 1/1 Running 0 5m
- Delete the multicluster-observability-operator CR:
$ kubectl -n open-cluster-management-observability delete -f config/samples/observability_v1beta2_multiclusterobservability.yaml
- Delete the multicluster-observability-operator:
$ make undeploy
- Delete the minio service:
$ kubectl -n open-cluster-management-observability delete -f observability-e2e-test/cicd-scripts/e2e-setup-manifests/minio
- Delete the
open-cluster-management-observability
namespace:
$ kubectl delete ns open-cluster-management-observability