Homer

Homer is a log parsing and queueing program written to parse apache logs and send them to an AMQP server as a JSON object.

Overview

Prerequisites

List of prerequisites for building and running the server:

Go compiler - for homer.go
libsqlite3-dev, gcc or llvm - for dependency go-sqlite3
Python - for the updatedb script
The IP database

Building the database

The database is composed of multiple files, from multiple sources. It's a combination of IP networks, country codes, city names, etc.

There's a helper script under the db directory that automates the process of building the database, and can be used regularly to update it as well. Because it downloads multiple files and process them, it might eventually fail.

It's a Python script called updatedb that generates ipdb.sqlite:

$ cd db
$ ./updatedb
... will download files and process them to generate ipdb.sqlite
$ file ipdb.sqlite
ipdb.sqlite: SQLite 3.x database

This service includes GeoLite data created by MaxMind, available from maxmind.com.

If you wish, there is an example crontab entry for weekly updating the db in the homer.cron file.

Build and run

Make sure the Go compiler is installed and $GOPATH is set. Install dependencies first:

go get github.com/mattn/go-sqlite3

Use either go run homer.go or go build; ./homer to compile and run homer.

Apache Config

You can setup homer to work with apache through its logs. If look at homer.conf you will see an example apache conf file.

AMQP Config

To change the AMQP settings simply edit the homer.cfg file.

Pointing to the cfg on execution

Use the -c option with homer to point to a config file as a full pile when running:

homer -c /path/to/homer.cfg

flippmoke / homer