Giters

VinodKandula / springboot-elk-prometheus-grafana

The goal of this project is to implement a Spring Boot application, called movies-api, and use Filebeat & ELK Stack (Elasticsearch, Logstash and Kibana) to collect and visualize application's logs and Prometheus & Grafana to monitor application's metrics.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

springboot-elk-prometheus-grafana

The goal of this project is to implement a Spring Boot application, called movies-api, and use Filebeat & ELK Stack (Elasticsearch, Logstash and Kibana) to collect and visualize application's logs and Prometheus & Grafana to monitor application's metrics.

Note: In kubernetes-minikube-environment repository, it's shown how to deploy this project in Kubernetes (Minikube)

Application

  • movies-api

    Spring Boot Web Java application that exposes a REST API for managing movies. Its endpoints are displayed in the picture below.

    movies-api

Prerequisites

Start Environment

  • Open a terminal and inside springboot-elk-prometheus-grafana root folder run

    docker-compose up -d

  • Wait a bit until containers be Up (healthy) (except filebeat). In order to check it run

    docker-compose ps

Running application with Maven

  • Open a terminal and make sure you are inside springboot-elk-prometheus-grafana folder

  • Run the following command

    ./mvnw clean spring-boot:run --projects movies-api

    Note: If you want to change to "non-json-logs" (maybe during development it's useful), run

    ./mvnw clean spring-boot:run --projects movies-api -Dspring-boot.run.jvmArguments="-Dspring.profiles.active=non-json-logs"

Running application as Docker container

  • Build Docker image

    • In a terminal, make sure you are inside springboot-elk-prometheus-grafana root folder
    • Run the following script to build the image
      • JVM 
        ./docker-build.sh
      • Native (it's not implemented yet) 
        ./docker-build.sh native

  • Environment variables

    Environment Variable Description
    MYSQL_HOST Specify host of the MySQL database to use (default localhost)
    MYSQL_PORT Specify port of the MySQL database to use (default 3306)

  • Start Docker containers

    • Finally, run the container 
      docker run -d --rm --name movies-api \
  -p 8080:8080 -e MYSQL_HOST=mysql \
  --network=springboot-elk-prometheus-grafana_default \
  ivanfranchin/movies-api:1.0.0

Application & Services URLs

  • movies-api

    movies-api Swagger is http://localhost:8080/swagger-ui.html

  • Prometheus

    Prometheus can be accessed at http://localhost:9090

    prometheus

  • Grafana

    Grafana can be accessed at http://localhost:3000

    • In order to login, type admin for both username and password
    • You can skip the next screen that ask you to provide a new password
    • Click General / Home on the top
    • Click movies-api-dashboard

    grafana

  • Kibana

    Kibana can be accessed at http://localhost:5601

    Configuration

    • Access Kibana website
    • Click Explore on my own
    • In the main page, click the "burger" menu icon and, then click Discover
    • Click Create index pattern button
    • In the Index pattern name field, set filebeat-* and click > Next Step button
    • In the Time field combo-box, select @Timestamp and click Create index pattern
    • Click the "burger" menu icon again and then click Discover to start performing searches

    kibana

  • Elasticsearch

    Elasticsearch URL is http://localhost:9200

    Useful queries

    # Check ES is up and running
curl localhost:9200

# Check indexes in ES
curl "localhost:9200/_cat/indices?v"

# Check filebeat index mapping
curl "localhost:9200/filebeat-*/_mapping"

# Simple search
curl "localhost:9200/filebeat-*/_search?pretty"

Shutdown

  • Stop application

    • If the application was started with Maven, go to the terminals where it is running and press Ctrl+C
    • If the application was started as a Docker container, run the command below 
      docker stop movies-api

  • To stop and remove docker-compose containers, network and volumes, make sure you are in springboot-elk-prometheus-grafana root folder and run

    docker-compose down -v

Reference

About

The goal of this project is to implement a Spring Boot application, called movies-api, and use Filebeat & ELK Stack (Elasticsearch, Logstash and Kibana) to collect and visualize application's logs and Prometheus & Grafana to monitor application's metrics.

Languages

Language:Java 94.3%Language:Dockerfile 3.7%Language:Shell 2.1%