Giters

Significant-Gravitas / AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

Home Page:https://agpt.co

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

use proxy for selenium

amirdaaee opened this issue · comments

commented

Duplicates

  • I have searched the existing issues

Summary 💡

Currently, the read_webpage function within web_selenium command lacks built-in support for using proxy servers. This feature request proposes adding functionality to enable users to specify a proxy server when utilizing the read_webpage function.

Examples 🌈

No response

Motivation 🔦

  • Scraping geo-restricted websites: By leveraging proxies located in different regions, users can access and scrape data from websites that might be restricted based on geographical location.

  • Enhanced privacy: Proxies can anonymize the user's IP address, potentially improving privacy while scraping data.

  • Rotational scraping: Utilizing a pool of proxies allows for distributing scraping requests across different IP addresses, reducing the risk of being flagged by websites.

commented

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

commented

This issue was closed automatically because it has been stale for 10 days with no activity.