Multiple Elastic Search node guide

[nyrm]

Hello!

My kuiper VM has 90b of ram assigned to it, wanted to know what the best way is to add a second ES node.

"If you want to use RAM more than 64GB to increase Elasticsearch performence, it is recommended to use multiple nodes for Elasticsearch cluster instead in different machines"

I would have to edit the docker-compose.yaml right?

Would you be able to provide assistance in for the best way to add? i followed a guide to adding a es02 node in the yaml file and i don think i got it to work quite right

[nyrm]

es02:

    build: 
        dockerfile: ./Dockerfile.prod.es01
        context: ./
        network: host

    image: dfirkuiper/kuiper_es01:${GIT_KUIPER_ES01_VERSION}
    container_name: kuiper_es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01
      - cluster.initial_master_nodes=es01
      - "ES_JAVA_OPTS=-Xms28g -Xmx28g"
      - FLASK_IP=flask
      - ES_IP=es01
      - bootstrap.memory_lock=true
    
    volumes:
      - ./elasticsearch02:/usr/share/elasticsearch/data:rw

    env_file:
        - ./.env
    networks:
        kuiper:
            ipv4_address: 172.30.250.19

[nyrm]

This is what ive added in the file for now

[saleh muhaysin]

i tried before with Elasticsearch cluster and it worked,
but my cluster was in multiple VMs (kuiper, cluster-master, cluster-data1, cluster-data2)
not sure if you will get the benefit of the jvm limitation if you run all cluster nodes in the same VM.

you need to change the ES_IP environment variable from the docker-compose.yaml file and .env file with the ip address of the cluster master node IP

[nyrm]

Thanks for the info! tried it out and wasnt working for me, i think i will just stick to the one node on my ubuntu VM