crab86 / grafana-plei

Grafana bundled with (P)rometheus, (L)ogstash,(E)lasticsearch,(I)nfluxDB to play around.

grafana-plei

Grafana bundled with (P)rometheus, (L)ogstash,(E)lasticsearch & (I)nfluxDB using Docker to play around. Simply start the whole stack with Docker-Compose (docker-compose up). No additinal agents are needed, all the metrics and logs in all dashboards are based up on internal statistics of the used databases and tools itself.

The PLEI-stack is based on the following official Docker images:

• Grafana
• Prometheus
• Logstash
• Elasticsearch
• InfluxDB
• cAdvisor

Note: cAdvisor is not started per Default (commented out in docker-compose.yml). So if you would like to have additional metrics to play around with simply uncomment the cAdvisor statement in docker-compose.yml. cAdvisor then pushes its metrics to InfluxDB (Grafana datasource already pre-provisioned) and prometheus scrapes them aswell. You would then need to add additional Dashboards on your own to build some cAdvisor Graphs.

Pre-provisioned Datasources in Grafana

Grafana comes pre-provisioned with the following datasources:

• Elasticsearch
• Elasticsearch-monitoring
• Elasticsearch-logstash-monitoring
• InfluxDB_internal
• InfluxDB_cadvisor
• Prometheus

Pre-provisioned Dashboards in Grafana

Moreover the following Dashboards are also pre-provisioned to Grafana when starting the PLEI-stack:

Dashboards with Elasticsearch as datasource:

• Elasticsearch - Container Logs (Note: All containers send their logs to Logstash via syslog driver. The logs can be viewed in this dashboard)
• Elasticsearch Monitoring based on X-Pack stats
• Logstash Monitoring based on X-Pack stats - repeat by node
• Logstash Monitoring based on X-Pack stats - grouped by node

Dashboards with InfluxDB as datasource:

• InfluxDB - InfluxDB internal metrics (based on 421)

Dashboards with Prometheus as datasource:

• Prometheus - Prometheus Stats (imported from Datasource)
• Prometheus - Prometheus 2.0 Stats (imported from Datasource)
• Prometheus - Grafana Metrics (imported from Datasource)

Viewable Metrics and Logs

Elasticsearch & Logstash

Elasticsearch offers X-Pack statistics to monitor Elasticsearch and Logstash out-of-the-box. You simple need to enable it via elasticsearch.yml / logstash.yml config files. The statistics are written to special elasticsearch indices and datasources+dashboards for those indices are already pre-provisioned when starting the stack. For additional Information please refer to the official documentation:

Elasticsearch Monitoring

Logstash Monitoring

In addition all used containers publish their logs to logstash's syslog input. All the logs can be viewed in a pre-provisioned dashboard aswell.

InfluxDB

InflxuDB offers internal statistics as described in the official documentation:

InfluxDB Internal monitoring

A datasource+dashboard using this internal database is already pre-provisioned when starting the PLEI stack.

Prometheus

Prometheus and Grafana expose /metrics endpoints which are scraped using jobs "prometheus" and "grafana". Moreover cAdvisor also exposes metrics which are scraped with a job "cAdvisor" although cAdvisor container is disabled by default, see Note.

The prometheus datasource is already pre-provisioned when starting the PLEI stack. The datasource also comes with dashboards, those are pre-provisioned as well.

Requirements

Host setup

1. Install Docker
2. Install Docker Compose
3. Clone this repository

Usage

Bringing up the stack

Start the PLEI stack using docker-compose:

$ docker-compose up

Grafana can then be accessed via http://localhost:3000 with a web browser (default login is admin:admin). Elasticsearch typically needs some time to completely start (around 2min.) so the dashboards with Elasticsearch as datasource may not be ready immediately.

By default, the PLEI stack exposes the following ports:

• 3000: Grafana
• 5140: Logstash Syslog UDP input
• 8086: InfluxDB HTTP
• 9090: Prometheus server
• 9200: Elasticsearch HTTP
• 9600: Logstash monitoring