Table of Contents
About The Project
I've seen multiple projects out there in GitHub, that are crawlers for the deep web, but most of them did not meet my standards of OSINT on the deep web. So I decided to create my own deep web OSINT tool.
This tool serves as a reminder that the best practices of OPSEC should always be followed in the deep web.
The author of this project is not responsible for any possible harm caused by the usage of this tool.
What does it do?
Prying Deep crawls dark/clear net platforms and extracts as much intelligence as possible effectively.
⚠️ PhoneNumber module is currently not working.
Please do not use it as of now.
Note:
Getting Started
Prerequisites
Before you can use our OSINT tool, please ensure you have the following dependencies installed:
-
Docker: (optional)
- You can download and install Docker by following the official installation instructions for your specific operating system:
- Docker Installation Guide.
-
Go: (required)
-
PostgresSQL: (required if you don't use docker)
- Make sure your .yaml environment variables match the environment in
docker-compose.yaml
- PostgreSQL Installation
- Make sure your .yaml environment variables match the environment in
Binary Installation
- Install the binary via:
go install -v github.com/iudicium/pryingdeep/cmd/pryingdeep@latest
- Run the following command:
pryingdeep install
- Adjust the values inside the config folder to your needs.
Manual Installation
-
Clone the repo:
git clone https://github.com/pryingbytez/pryingdeep.git
-
Adjust the values in the .yaml configuration either through flags or manually.
Database
,logger
,tor
all require manual configuration.
You will need to read Colly Docs. Also, refer to Config Explanation
- Build the binary via
go build
-> directory:cmd/pryingdeep
go build cmd/pryingdeep/pryingdeep.go
-> root directory, binary will also be there.
Docker
To start run pryingdeep inside a docker container use this command:
docker-compose up
Config
Read more each parameter here: config
Tor
Read more about building and running our tor container here: Tor
Usage
Crawler:
./pryingdeep crawl
Exporter:
./pryingdeep export
Roadmap
- Fix the phoneNumber module, get it to at least 50%
- Implement file identification and search
Contributing
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
License
Distributed under the GPL-3.0 license. See LICENSE
for more information.