ES Kit

A toolkit for ES cluster management and load testing

Docker

Pull Image

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.4.2

Single node cluster

docker run -p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
docker.elastic.co/elasticsearch/elasticsearch:7.4.2

SSH in to the docker container

docker exec -it es01 /bin/bash

Tools

Curator: maintenance tool link
Python: official py client link)
Ansible: automation link
Dump: tool, requires node js link

Get info from cluster

Build info:

curl -X GET "localhost:9200/_xpack?categories=build,features&pretty"

Cluster health

curl -X GET "localhost:9200/_cluster/health?wait_for_status=yellow&timeout=50s&pretty"

Cat API: (nodes, indices, cluster, segments, etc...)

Enable Slow Queries Log

PUT /index/_settings
{
    "index.search.slowlog.threshold.query.warn: 1s",
    "index.search.slowlog.threshold.query.info: 500ms",
    "index.search.slowlog.threshold.query.debug: 1500ms",
    "index.search.slowlog.threshold.query.trace: 300ms",
    "index.search.slowlog.threshold.fetch.warn: 500ms",
    "index.search.slowlog.threshold.fetch.info: 400ms",
    "index.search.slowlog.threshold.fetch.debug: 300ms",
    "index.search.slowlog.threshold.fetch.trace: 200ms"
}

Get Mappings for Index

To retrieve mapping definitions for indices in a cluster

curl -X GET "localhost:9200/<index>/_mapping?pretty"

Get mapping for specific field

curl -X GET "localhost:9200/my-index/_mapping/field/<field>?pretty"

Config

3 settings:

elasticsearch.yml for configuring Elasticsearch
jvm.options for configuring Elasticsearch JVM settings
log4j2.properties for configuring Elasticsearch logging

See Scaling for more info

Caching

Shard request cache

Most queries that use now (see Date Math) cannot be cached

Cache can be expired manually

curl -X POST "localhost:9200/kimchy,elasticsearch/_cache/clear?request=true&pretty"

Enabling and disabling caching per request edit (overrides index setting)

curl -X GET "localhost:9200/my_index/_search?request_cache=true&pretty" -H 'Content-Type: application/json' -d'
{
  "size": 0,
  "aggs": {
    "popular_colors": {
      "terms": {
        "field": "colors"
      }
    }
  }
}
'

Cat API

GET /_cat
    =^.^=
    /_cat/allocation
    /_cat/shards
    /_cat/shards/{index}
    /_cat/master
    /_cat/nodes
    /_cat/indices
    /_cat/indices/{index}
    /_cat/segments
    /_cat/segments/{index}
    /_cat/count
    /_cat/count/{index}
    /_cat/recovery
    /_cat/recovery/{index}
    /_cat/health
    /_cat/pending_tasks
    /_cat/aliases
    /_cat/aliases/{alias}
    /_cat/thread_pool
    /_cat/plugins

cnnrrss / es-kit