|
Note
|
How metrics is installed has changed from using the deployer pod to using ansible. Please see the OpenShift Metrics installation instructions for more information. |
origin-metrics
About
Origin Metrics is designed to gather container, pod and node metrics from across an entire OpenShift cluster. These metrics can then be viewed in the OpenShift Console or exported to another system.
It achieves this goals via these main components:
Heapster
Heapster gathers the metrics from across the OpenShift cluster. It retrieves metadata associated with the cluster from the master API and retrieves individual metrics from the /stats endpoint which is exposed on each individual OpenShift node.
It gathers system level metrics such as CPU, Memory, Network, etc
Heapster will then send these metrics to Hawkular Metrics and expose a REST endpoint that the horizontal pod autoscaler (HPA) uses.
Hawkular Metrics
Hawkular Metrics is the metric storage engine from the Hawkular project. It provides means of creating, accessing and managing historically stored metrics via an easy to use json based REST interface.
Hawkular Metrics uses Cassandra as its metric datastore.
It receives metrics from the Heapster component and is also used to expose metrics to the console and other third party systems.
Cassandra
Cassandra is the data store for Hawkular Metrics. It stores and persists all the metrics coming in from Hawkular Metrics.
Hawkular OpenShift Agent
The Hawkular OpenShift Agent is a component which can be used to gather application level metrics coming from pods. This allows pods themselves to expose metrics that they wish to be collected.
This component runs as a deamon set across the OpenShift cluster and is responsible for gathering application level metrics from each pod running on that node.
By default we deploy the agent to the default project. By deploying to the default project, the agent will be able to monitor all pods even if running with the ovs_multitenant plugin.
|
Note
|
The Hawkular OpenShift Agent is currently in Tech Preview. |
Before You Begin
The Metrics components requires that OpenShift is properly installed and configured. How to properly install and configure OpenShift is beyond the scope of this document. Please see the OpenShift documentation on how to properly install and setup your system.
The metrics install requires integation with the OpenShift cluster and if there are installation or configuration problems with your OpenShift cluster you may encounter them when tyring to running metrics. Some of the more common issues are checked in the deployer pod and will give you an error message about your installation.
Please see the troubleshooting guide for a complete list of things to watch out for.
Installing
The OpenShift documentation describes how to install metrics.
|
Note
|
The old way of deploying metrics which rely on using the deployer pod is now deprecated. For the old instructions please see the deployer installation page. |
Deploying the Hawkular OpenShift Agent
A few steps are required to deploy the Hawkular OpenShift Agent into OpenShift.
You will first need to create the ConfigMap that the Agent uses to configure itself:
$oc create -f hawkular-agent/hawkular-openshift-agent-configmap.yaml -n default
You will then need to process the Agent’s template and deploy its components:
$oc process -f hawkular-agent/hawkular-openshift-agent.yaml | oc create -n default -f -
Finally you will need to grant the `hawkular-openshift-agent’s service acount the proper permissions:
$oc adm policy add-cluster-role-to-user hawkular-openshift-agent system:serviceaccount:default:hawkular-openshift-agent
Once this is completed, the Hawkular OpenShift Agent should be deployed into the default project.
If you are running the latest OpenShift, you should be able to verify that the agent is functioning by seeing extra metrics showing up under the agent’s pod metric page.
For instructions on how to expose custom metrics on your own pod and for more information about the agent itself, please the the Hawkular OpenShift Agent project.
Accessing Metrics Directly
If you wish to access and manage metrics directly, you can do so via the Hawkular Metrics REST API. This will allow you to directly access the raw metrics data and export it for use in your own customized systems.
For more information please see the Hawkular Metrics page.
Accessing the Hawkular Metrics Python Client
The Hawkular Metrics pod has been configured so that the Hawkular Python client is installed and given admin priviledges by default. This will allow you to check and query metrics directly from within the Hawkular Metric pods.
The command to run the client is client and can be run from the terminal within the OpenShift console or via the oc exec command.
Please see the Hawkular Metrics python client project for more information.
Accessing Heapster Directly
The Heapster deployed as part of origin-metrics is configured to be only accessible via the API Proxy. Access will require cluster-admin privileges or modifications to your policy to allow another user access to this API Proxy endpoint.
For example, to reach the Heapster metrics endpoint, you would need to access it by doing something like:
$ curl -H "Authorization: Bearer XXXXXXXXXXXXXXXXX" \
-X GET https://${KUBERNETES_MASTER}/api/v1/proxy/namespaces/openshift-infra/services/https:heapster:/api/v1/model/metrics
For more information about Heapster and how to access its APIs, please refer the Heapster project.
Cleanup
If you wish to undeploy and remove everything deployed by the deployer, the follow commands can be used:
$ oc delete all,secrets,sa,templates --selector=metrics-infra -n openshift-infra
|
Note
|
The persistent volume claim will not be deleted by the above command. If you wish to permanently delete the data in persistent storage you can run |
If you wish to remove the deployer’s components themselves
$ oc delete sa,secret metrics-deployer -n openshift-infra
Docker Containers
All the docker images for Origin Metric components are available at docker hub and there should not be a need to build these directly.
If you wish to build your own images or hack on the project. Please see the build instructions.
Known Issues
Please see the known issues page in the documentation.