htrgouvea / harpoon

[W.I.P] An ecosystem of crawlers for detecting: leaks, sensitive data exposure and attempts exfiltration of data

Harpoon

An ecosystem of crawlers for detecting: leaks, sensitive data exposure and attempts exfiltration of data

---

Summary

⚠️ Warning: Harpoon is currently in development, you've been warned :) and please consider contributing!

This project is summarized in several crawlers that constitute a single ecosystem, that monitor certain channels such as: Github, Bing, Pastebin and iHaveBeenPwned? in order to perform data leak detection, exposed sensitive files and data exfiltration attempts.

---

How it works

• You can click here to see a diagram of the database and also a catalog on them.

---

Download and setup

  # Download
  $ git clone https://github.com/GouveaHeitor/uranus && cd uranus

  # Building and starting MariaDB Database
  $ docker build -t uranus-database ./rest-server/database/
  $ docker run -d -p 3306:3306 --name database -e MARIADB_ROOT_PASSWORD=mypassword uranus-database

  # Building and starting the REST API
  $ docker build -t uranus-rest-server ./rest-server.
  $ docker run -d -p 80:80 --name rest-server uranus-rest-server

  # Building all crawlers/workers containers
  $ docker build -t bing-crawler ./crawlers/bing/
  $ docker build -t email-notify ./workers/email-notify

  # Running all crawlers/workers containers
  $ docker run -d --name bing bing-crawler
  $ docker run -d --name email-notify email-notify

---

Contribution

Your contributions and suggestions are heartily ♥ welcome. See here the contribution guidelines. Please, report bugs via issues page and for security issues, see here the security policy. (✿ ◕‿◕)

---

License

This work is licensed under MIT License.