branning / nginx-minder

Watch nginx logs and periodically report status code counts in statsd format

nginx-minder

Watch nginx logs and periodically report status code counts in statsd format.

About

The nginx-minder program is distributed with a Dockerfile and configuraiton files to run it to monitor the logs from a group of Nginx instances deployed in a high-availability Kubernetes cluster.

What does it do?

On each nginx instance, the local access log file at /var/log/nginx/access.log is followed, and the HTTP status code of each route accessed on the server is noted. For any routes that produce status codes in the 500-599 range (fatal error codes), the route that produced the code is also noted. Every 5 seconds, the total count of 200, 300, 400, 500 status codes is printed in statsd-compatible log messages, followed by a list of routes that produced 500 status codes with the code count.

Deployment

1. Build & push

   The containers are built and stored in Docker Hub. In order to build the containers, you can run make. In order to push new
   containers that will be pulled by the Kubernetes deployment, you must first login to the Docker Hub repository:

    docker login --username=branning                                             
   

   After logging in, you can push new containers with make push.

2. Create pods and deployment

   First you'll need a Kubernetes cluster that you can control using kubectl. Use minikube for local testing and deployments. Once installed, run minikube start to use the default
   virtualbox provisioner, or minikube start --vm-driver=xhyve for a
   MacOS system with the xhyve virtualization layer installed.

   To deploy and create pods, run kubectl create -f k8s/nginx-minder-deployment.yaml. It will take perhaps 20-60 seconds to pull the 2 containers from Docker Hub. To check the pod status, run kubectl get pods. When you see 2/2 pods are ready, you can check the output of the nginx-minder process by calling
   kubectl logs <pod-name> nginx-minder. The pod names are automatically
   generated by the deployment, you can see them in the output of kubectl get pods.

3. Test it out

   You now have a kubernetes deployment, and requisite service and 2 pods each with 2 containers. To visit the service, find out the IP of your local minikube with minikube ip, and the port used by the service with kubectl get service. Then you can visit that IP at the service's port to hit the nginx server.