jaisonora / bablo

Bablo is a recursive crawler written in Golang

Bablo (Бабло)

Bablo (Бабло) is a web scraping tool designed to explore and collect URLs from a specified starting URL. It utilizes the "github.com/gocolly/colly/v2" package.

Table of Contents

• Features
• Usage
• Installation
• Dependencies
• License
• Author
• Contributing

Features

• Recursive crawling
• Multithreading

Usage

1. Clone or Download: Clone this repository or download the code to your local machine.
2. Install Golang: Install the latest Golang from https://go.dev/dl/. For example:

   wget https://go.dev/dl/go1.21.4.linux-amd64.tar.gz
   rm -rf /usr/local/go && tar -C /usr/local -xzf go1.21.4.linux-amd64.tar.gz
   export PATH=$PATH:/usr/local/go/bin
   go version
   

3. Install colly: Run the following commands:

   go mod init bablo
   go mod tidy
   go get github.com/gocolly/colly/v2@latest
   

4. Run the tool: Run the tool using the following command:

   go run main.go -t <thread-count> https://example.com
   

5. Automation: For automating this tool with others check - https://github.com/grozdniyandy/hackerlines

Installation

You can either check the "Usage" or download already compiled code from "releases".

Dependencies

This code uses the Go standard library and one external dependency github.com/gocolly/colly/v2.

License

This code is released under the MIT License.

Author

Bablo is developed by GrozdniyAndy of XSS.is.

Contributing

Feel free to contribute, report issues, or suggest improvements by creating pull requests or issues in the GitHub repository. Enjoy using this crawler!