jnh5y / geo-env

Docker containers with Apache Accumulo and Apache Spark environment.

Docker Geo environment.

Docker containers with prepared environment to run GeoTrellis, GeoMesa, and GeoWave jobs. These images will create a set of containers, running in a distributed fashion, on a single machine. In practice, this requires being careful to ensure that enough memory is available for all images.

Environment

• Hadoop (HDFS + YARN) 2.7.1
• ZooKeeper 3.4.6
• Accumulo 1.6.4 / 1.7.0 (the specific version to use is configurable)
• Spark 1.5.2

Repository short description (index of ReadMe docs)

Base images:

• base image

  • Contains a Dockerfile to build an image with Hadoop, ZooKeeper, Accumulo and Spark installed (but not configured).
  • Available on Dockerhub:
    • geo-base

• serf image

  • Contains a Dockerfile to build an Ubuntu 14.04 image with serf.
  • Available on Dockerhub:
    • serf

Concrete images: Note: It is possible to have a multinode (n-node) cluster with any number of Zookeeper instances.

• single-node image

  • Contains a ZooKeeper node, working in a singlenode mode.
  • Available on Dockerhub:
    • geo-master-sn
    • geo-slave-sn

• two-node image

  • Available on Dockerhub:
    • geo-master-twn
    • geo-slave-twn

• three-node image

  • Available on Dockerhub:
    • geo-master-thn
    • geo-slave-thn

GeoTrellis, GeoMesa, and GeoWave:

• install directory
  • Contains scripts to install GeoTrellis, GeoMesa and GeoWave into cluster and to run test examples, to be sure that cluster and library operating correct.

Build a multinode cluster

A more detailed description how to run and to build containers can be found in each image directory.

• Build serf container

  • cd serf; ./build.sh

• Build base container

  • cd base; ./build.sh

• Build n-node master and slave containers

  • cd n-node; ./build.sh

Sart the n-container cluster.

• cd n-node; ./start-cluster.sh --nodes=nn # nn >= n

License

• Based on a repository: https://github.com/alvinhenrick/hadoop-mutinode
• Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0