Modular outputs
eth0izzle opened this issue · comments
shhgit should be able to output to different sources and formats, i.e. csv, json, a postgres database, UDP, elasticsearch, etc. We should take a modular approach for extensibility, i.e. struct embedding
I'd like to take a crack at this if you are willing to let me. I should have something for you to review early next week.
Was having a similar conversation recently as I have a super basic modular framework that's similar to IFTTT meant for security teams to build their own modules around.
As I'm not trying to hijack this thread with advertising my own project that I haven't been updating publicly for a bit I'll lend a possible solution that seems to align with the latest pull request on this comment.
Using a MessageBroker like RabbitMQ would allow for a super modular system for publishing events to as many different outputs as you'd like, all of which would be programming language agnostic and could even be Lambda functions if you wanted. Having sshgit publish to a fan-out exchange (One-to-Many) would allow the bulk of the output processing to be offloaded to consumers to handle however they see fit as well as keep the general codebase of sshgit more or less untouched once a standardized event message format is agreed on.
Having a fan-out exchange makes things super useful if for example a user wanted to store data in a database, send a slack notification, and (for example) grab AWS keys out of an event to enumerate if a key is working and what all the key gives access to (which can then be stored or alerted on or whatever). sshgit would send a single message to the MessageBroker and that same message could be handled by as many different modules as the user wants.
Just my idea of a good way to handle this after looking over the PR and seeing that a basic MessageBroker was being implemented.