Zila-itc / botasaurus

The All in One Web Scraping Framework

✨ Botasaurus Framework 🤖

🚀 Swiss Army Knife for Bot Development 🤖

(Programming Language - Python 3)

Botasaurus - Your Ultimate Selenium-Based Bot Development Framework

🤖 Hi, I'm Botasaurus, a Selenium-based bot development framework designed to help you become a 10x Bot Developer!

🔍 What is Botasaurus and How Can it Help Me Become a 10x Bot Developer?

Think of Botasaurus as a supercharged version of Selenium. If you know Selenium, you're off to a great start because Botasaurus is a framework for Selenium with goodies like:

1. Account Generation

• My Account Generation Module enables you to create humane user accounts effortlessly. This is particularly handy when you need multiple accounts while evading detection.

2. Temporary Email

• Need to verify those accounts? I've got you covered with an inbuilt Temporary Email Module.

3. Simple Configuration

• Configuring profiles, user agents, and proxies can be a headache in Selenium. But with Botasaurus, it's as easy as pie!

4. Parallel Bots

• Run multiple bots in parallel and turn hours of automation into mere minutes with Botasaurus.

5. Easy Debugging

• Debugging made painless! When a crash occurs due to an incorrect selector, etc., Botasaurus pauses the browser instead of closing it, facilitating on-the-spot debugging.

• Capture important details like screenshots, page HTML, error logs, and execution time for easy debugging and generating time estimates of how long to keep the Bot Running.

6. Run Bots at Google's Scale with Kubernetes

• Seamlessly integrate Botasaurus with Kubernetes to run your bots at Google's Scale. Easily control the number of bots and when to launch them via a simple UI interface.

7. Turbocharge Your Web Scraping

• Say goodbye to redundant requests with our powerful caching feature. Botasaurus allows you to cache web scraping results, ensuring lightning-fast performance on subsequent scrapes.

8. Supercharged Selenium Driver

• Elevate your Selenium game with Botasaurus. We've added Anti-Bot Detection features and added shortcut methods for common operations, all designed to save your valuable time.

9. Persist 1000 Profiles in 1 MB

• Forget about heavy 100 MBs Chrome profiles. Use the Tiny Profile feature to save cookies in just a few KBs.

10. Evade IP Suspensions like Ninja Hatori

• Using a single IP to create accounts and getting suspended?

No problem! Ditch expensive $15/GB Residential Proxies and use the FreeProxy Module to connect to the Tor Network and access a vast pool of unique Tor IPs all for free.

So, if you're comfortable with Selenium and want to level up your bot development game with easy debugging, simple configuration, parallel execution, and temporary emails, then you're in for a treat. Botasaurus is like Selenium on steroids, and you'll love building Bots with it!

So, if you're comfortable with Selenium and want to level up your bot development game with easy debugging, simple configuration, parallel execution, and temporary emails, then you're in for a treat. Botasaurus is like Selenium on steroids, and you'll love building Bots with it!

🚀 Getting Started with Botasaurus

Let's quickly grasp how Botasaurus works with a simple example.

In this simple example, we'll walk through the process of scraping the heading text from https://www.omkar.cloud/.

Step 1: Install Botasaurus

Start by installing Botasaurus with the help of this command:

python -m pip install botasaurus

Step 2: Set Up Your Botasaurus Project

1. Create a directory for our Botasaurus project and navigate to it:

mkdir my-botasaurus-project
cd my-botasaurus-project
code .  # Optionally, open the project in VSCode

Step 3: Write the Scraping Code

Within your project directory, create a Python script named main.py and paste the following code into main.py:

from botasaurus.launch_tasks import launch_tasks
from botasaurus import *

# Define a custom scraping Task
class ScrapeHeadingTask(BaseTask):

    def run(self, driver: BotasaurusDriver, data):
        # Visit the Omkar Cloud website
        driver.get("https://www.omkar.cloud/")

        # Get the heading element text
        heading = driver.text("h1")
    
        # Return the data to be saved as a JSON file in output/all.json
        return {
            "heading": heading
        }

if __name__ == "__main__":
    # Launch the web scraping task
    launch_tasks(ScrapeHeadingTask)

Let's break down this code:

• We define a custom scraping task class named ScrapeHeadingTask.

class ScrapeHeadingTask(BaseTask):

• Inside the run method, we are automatically passed a Selenium driver by Botasaurus.

    def run(self, driver: BotasaurusDriver, data):

• In the run method:
  • We visit Omkar Cloud
  • Extract the heading text
  • Finally, return data to be saved as JSON and CSV files.

    driver.get("https://www.omkar.cloud/")

    # Get the heading element text
    heading = driver.text("h1")
  
    # Return the data to be saved as a JSON file in output/all.json
    return {
        "heading": heading
    }

• Lastly, we launch the scraping task.

if __name__ == "__main__":
    # Launch the web scraping task
    launch_tasks(ScrapeHeadingTask)

Step 4: Run the Scraping Task

Now, let's run the bot:

python main.py

After running, the script will:

• Launch Google Chrome
• Visit omkar.cloud
• Extract the heading text
• Automatically save it as output/finished.json.

🚀 Deep Dive into Botasaurus

Power of Bots is Immense, A Bot

• Can apply on your behalf to Linkedin Jobs 24 Hours
• Scrape Phone Number of Thousands of Buisnesses from Google Maps to sell your Products to.
• Mass Message People on Twitter/LinkedIn/Reddit for selling your Product
• Sign Up 100's of Accounts on MailChimp to send 50,000 (500 emails * 100) Emails all for free

So, Ready to unlock the power of Bots?

📚 Read the Tutorial here and embark on your journey of bot mastery! 🚀

Sponsors

Special Sponsor

Love It? Star It! ⭐