monbox

Cloud monitoring service on AWS utilizing Spring Boot, Spring Cloud, AWS SDK, Prometheus, Grafana, Docker and Terraform. Heavily borrowed from Hygieia's AWS Cloud collector but employs Redis (or ElastiCache) for persistence.

This is a Spring Boot application.

Prerequisites

An AWS account. You may follow instructions to create one here.
aws-cli 1.11.34 or better
Docker Toolbox; or docker, docker-machine and docker-compose are required
Java JDK 1.8.0_111 or better
Maven 3.3.9 or better

Prepare AWS CLI

See

You must configure a default profile, e.g.,

$ aws configure --profile default

How to obtain the source

You'll use a git client.

with HTTPS

git clone https://github.com/fastnsilver/monbox.git

with SSH

git clone git@github.com:fastnsilver/monbox.git

How to build

with Maven

$ mvn clean verify

with Jenkinsfile

Pipeline support to be designed

Prepare to work with Redis

This service interacts with a Redis instance. Assumes instance is up-and-running at localhost (127.0.0.1). If you want to change that then you need to add the following argument (when attempting to run the service)

-Dspring.redis.host={redis.host}

where {redis.host} is the hostname or IP address of the Redis instance

You could start an instance of Redis with Docker using

docker run -p6379:6379 -d redis:3.2.6-alpine

To get the IP address of the Docker host

docker-machine ip {machine.name}

where {machine.name} is name of the Docker machine.

To tear down

docker ps
docker kill {container.id}
docker rm {container.id}

where {container.id} is the id of the running Redis container.

ElastiCache support

Off-cloud

From Accessing ElastiCache Resources from Outside AWS

for testing and development purposes only. It is not recommended for production use.

Running a local instance of this service requires that you provision a NAT instance or configure a NAT Gateway. See Comparison of NAT Instances and NAT Gateways.

Assign an Elastic IP to the NAT instance or NAT Gateway and use that IP address as the value for the Redis host.

On AWS

configure an ElastiCache cluster in an Amazon VPC

See (Amazon VPC) with ElastiCache

Each node within a cluster will have an endpoint addressable at

{node.name}.{cluster.id}.{region.id}.cache.amazonaws.com:6379

Once the cluster is configured and available make sure to set the Redis host environment variable.

This can be done e.g.,

when running service on an EC2 instance, single JVM

by supplying a command-line argument -Dspring.redis.host

when running as a task on an ECS cluster

by supplying an environment variable, SPRING_REDIS_HOST, within a task definition

How to Run

with Spring Boot

$ mvn spring-boot:run -Dspring.profiles.active=local -Djava.security.egd=file:/dev/./urandom

with Java

$ java -jar monbox-x.x.x-SNAPSHOT-exec.jar -Dspring.profiles.active=local -Djava.security.egd=file:/dev/./urandom

with Docker

Assuming you have installed VirtualBox, Docker Machine, Docker Compose and Docker.

If not, it's highly recommended (on a Mac) to install each via Homebrew with

brew tap caskroom/cask
brew install brew-cask
brew cask install virtualbox

brew install docker-machine
brew install docker-compose
brew install docker

The instruction below provisions a Docker host named dev with 2 CPU, 10Gb RAM and 20Gb disk space

docker-machine create --driver virtualbox --virtualbox-cpu-count "2" --virtualbox-disk-size "20000" --virtualbox-memory "10240" dev

You could also execute the following script which will perform the first step above on your behalf

./provision.sh {1}

where {1} above would be replaced with whatever you want to name your docker-machine

Caveat: You should have at least 12GB of memory and 25GB of disk space on your laptop or workstation.

To begin using it (e.g., where machine name was dev)

eval $(docker-machine env dev)

Lastly, to destroy your docker machine, you could execute

./destroy.sh {1}

where {1} above would be replaced with an existing docker-machine name

Caution! This will remove the VM hosting all your Docker images.

How to remote Debug

May be useful when you want to debug the service running on e.g., an EC2 instance.

See this Spring Boot Maven Plugin page for further details.

export SPRING_REDIS_HOST={elasticache.redis.endpoint}
mvn spring-boot:run -Drun.jvmArguments="-Xdebug -Xrunjdwp:server=y,transport=dt_socket,suspend=n,address=8000"

Docker notes

Build image

./build.sh

Publish image

Assumes proper authentication credentials have been added to $HOME/.m2/settings.xml. See:

Authenticating with Private Registries

mvn clean install -DpushImage

Pull image

TBD

Run image

./startup.sh

Running a local development environment

See Running localhost

On a Mac we cannot access running Docker containers from localhost.

After running docker-machine ip {env} where {env} is your instance of a docker-machine, add an entry in /etc/hosts that maps DOCKER_HOST IP address to a memorable hostname.

Work with image

Services are accessible via the Docker host (or IP address) and port

Service	Host Port	Container Port
Monbox Collector	80	8080
Redis	6379	6379
CAdvisor	9080	8080

Visit e.g., http://192.168.99.100/mappings

Stop image (and remove it)

./shutdown.sh

EC2 notes

See Getting Started guide. Minimum required instance type is t2.micro (which qualifies for free-tier).

Make sure to create a Key-pair and download the private key to a safe location. Also create an IAM Role with a ReadOnlyAccess policy and assign this role to the instance upon creation. The Security group should have TCP inbound ports 22, 8000 and 8080 open.

Then...

Connect to your EC2 instance with

 ssh -i /path/to/{your-private-key-filename}.pem ec2-user@{public-ip-address-of-instance}

Configure an additional YUM repo and install the following packages

 sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
 sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
 sudo yum install -y apache-maven java-1.8.0-openjdk-devel git

Set JAVA_HOME and PATH in .bashrc so that Java 8 is the default
- Change directories and open VI
```
 cd ~
 vi .bashrc
```
- Add the following lines at the end of the file, save, and exit VI
```
 export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64
 export PATH=$JAVA_HOME/bin:$PATH
```
  note: the minor version referenced above may be updated to a later version if one is available
Update session
```
 source ~/.bash_profile
```
Verify Maven is employing Java 8
```
 mvn -version
```
Clone

See Clone above. Choose HTTPS option.
Build
```
 cd monbox
 mvn clean verify
```

Test connection to ElastiCache

 nc -v {node.name}.{cluster.id}.{region.id}.cache.amazonaws.com 6379

or with Docker version of redis-cli

 sudo yum install -y docker
 sudo service docker start
 sudo docker sudo docker run -i -t prologic/redis-cli -h {node.name}.{cluster.id}.{region.id}.cache.amazonaws.com

Run

 export SPRING_REDIS_HOST=elasticache.redis.endpoint}
 java -jar target/monbox-x.x.x-SNAPSHOT-exec.jar

Test Endpoints

For test purposes only! Not recommended for large environments with 1000s of compute resources.

Request	Description
GET /cloudInstance?all
GET /cloudInstance/history?all
GET /volume?all
GET /subNetwork?all
GET /virtualNetwork?all

Working with Maven Site

Stage

mvn site site:stage -Pdocumentation

Publish

Assumes a gh-pages (orphan) branch has been set up in advance. In addition, appropriate authentication credentials have been declared in $HOME/.m2/settings.xml. See:

mvn scm-publish:publish-scm -Pdocumentation

fastnsilver / monbox