A simple console program that watches a given actively written-to w3c-formatted logfile and monitors HTTP traffic on a machine, alerts during periods of high volume, and provides some summative metrics.
This program is designed to be run in console. Command-line options are passed for both the logfile to watch (-f or --filename) and the critical threshold at which to alert for high traffic volume (-m or --max):
python logwatch.py -f access.log -m 100- Clone the repository:
$ git clone https://github.com/kaesackett/http-console.git- Create and activate a virtual environment in the same directory:
$ pip install virtualenv
$ virtualenv env
$ source env/bin/activate
- Install the required packages using pip:
(env)$ pip install -r requirements.txt- The script's current logic only follows, so unless the file you're trying to watch is actively growing, the stats dict is never populated (because it's populated only by new events as they occur, not events that already exist in the log). This will cause Python to throw a key error, as the stats dict has no keys.
- "Threading with Python? What about the Global Interpreter Lock? Why would you do such a thing?" - Right, that's a thing. In this case, I've got one thread to collect and manipulate the metrics (so, just fancy counting), and another to spit them out at regular intervals. I'm not doing anything here that requires any kind of precision. Give me a break. I literally wrote this in like an hour and a half.
- Add TTL to stats dict to expire old items
- Either retire on new write or
- Spawn a garbage collector thread to walk dict and cleanup
- Use a real DB
- Add a frontend (Web UI) that reads from DB
- Instead of watching file, wrap logging method