scraping-python

There are 21 repositories under scraping-python topic.

• ScrapeGraphAI / Scrapegraph-ai

  Python scraper based on AI

  ai-crawlerai-scrapingai-searchautomated-scrapercrawlerdata-extractionlarge-language-modelllmmarkdownragscrapingscraping-pythonweb-crawlerweb-crawlersweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
  Language:Python 21744

• lorien / awesome-web-scraping

  List of libraries, tools and APIs for web scraping and data processing.

  web-scrapingcaptcha-bypasscaptcha-recaptchacrawlingcrawling-frameworkcrawling-pythoncrawling-toolscrapingscraping-frameworkscraping-pythonscraping-toolwebscrapingcrawlerspider
  Language:Makefile 7423
• botasaurus

  omkarcloud / botasaurus

  The All in One Framework to Build Undefeatable Scrapers

  anti-botanti-detectioncloudflare-bypasscloudflare-scrapeanti-detectanti-detect-browserantidetect-browserundetectedundetected-chromedriverbypass-cloudflarepython-web-scraperpython-web-scrapingscraping-frameworkscraping-toolundetectableweb-scraping-pythonbot-detectionscraping-pythonweb-crawlingpython-scraper
  Language:Python 3217

• kaliiiiiiiiii / Selenium-Driverless

  a stealthy browser automation framework

  automationdetection-evasionpythonpython3reverse-engineeringscraping-pythontestingvulnerability-researchweb-scrapingwebdriverdriverless-chrome
  Language:Python 831

• scrapfly / scrapfly-scrapers

  Scalable Python web scraping scripts for +40 popular domains

  crawlingpythoncrawlerscrapingweb-scrapingweb-scraping-pythonantibotautomationcaptcha-bypasscrawling-pythondatascrapingproxiespython-scraperscraperscraping-pythonspidertwitter-scraperweb-crawlerwebscraperwebscraping
  Language:Python 749
• MetaDetective

  franckferman / MetaDetective

  🕵️ Unleash Metadata Intelligence with MetaDetective. Your Assistant Beyond Metagoofil.

  cybersecurityexifexif-data-extractionexif-metadatametadatametadata-informationmetadata-parsermetagoofilosintosint-emailosint-reconnaissanceosint-toolosint-toolspythonpython3scrapingscraping-pythonscraping-websecurity-toolsecurity-tools
  Language:Python 409

• lkuffo / web-scraping

  Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup

  scrapingscraping-pythonscraping-websiteswebscrapingscrapping-pythonselenium-webdriverselenium-pythonlxml-etreescrapyscrapy-crawlerscrapy-spiderbeautifulsoup4beautifulsoupseleniumweb-scraping
  Language:Python 378

• furas / python-examples

  Python examples from my answers on Stackoverflow and other short scripts.

  pythonexamplesscraping-pythonthreadingflasksocketguitkinter
  Language:Jupyter Notebook 187

• lobstrio / shadow-useragent

  Pick the most common user-agents on the Internet 👻

  useragentscrapingscraping-pythonuseragent-switch
  Language:Python 170

• fernandod1 / Instagram-to-discord

  Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

  instagram-scraperinstagraminstagram-photosinstagram-downloaderdiscorddiscord-botdiscordbotdiscordappinstagram-botmonitoring-scriptsmonitormonitorsscraperscrapingscrapperscraping-websitesscraping-pythonpythonpython3webhook-discord
  Language:Python 127

• hhuayuan / spiderbuf

  Spiderbuf 是一个专注于 Python 爬虫练习的网站。提供丰富的爬虫教程、爬虫案例解析和爬虫练习题。Python爬虫开发强化练习，在矛与盾的攻防中不断提高技术水平，通过大量的爬虫实战掌握常见的爬虫与反爬套路。 引导式爬虫案例 + 免费爬虫视频教程，以闯关的形式挑战各个爬虫任务，培养爬虫开发的直觉及经验，验证自身爬虫开发与反爬虫实力的时候到了。

  crawlerpythonspiderrequestsxpathspiderbufscrapingscraping-pythonscraping-webcrawler-pythoncrawlersscraping-datascraping-websitesseleniumcaptchacookiecookiesjs-reverse
  Language:Python 116

• thewebscraping / tls-requests

  TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

  tls-clientcloudflare-scraperweb-scraping-apicloudflare-bypasscf-clearanceanti-botanti-bot-detectionanti-bot-pagepython-tls-clientpython-crawlerpython-spiderweb-scraping-pythonpython-web-scraperpython-web-scrapingpython-web-crawlerweb-spiderscraping-pythonweb-crawler-pythonpython-scrapercrawling-python
  Language:Python 109

• Steelio / Telegram-Post-Scraper

  Python program designed to scrape posts from Telegram channels using HTTP requests and HTML parsing, rather than Telegrams API. This is useful, as selfbots are against Telegram's ToS.

  pythonscraping-pythontelegramtelegram-scrapertgtg-scrapingtelegram-parsertelegram-scraper-2022telegram-scraper-bottelegram-scraper-2023tele-img-downloadtelegram-post-stealertg-downloader-bottele-scraper-bottg-downloadtg-scraper-bot
  Language:Python 102

• ScrapingAnt / amazon_scraper

  Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

  scrapingscraping-apiscraping-websitesscraping-webscraping-pythonscraping-dataprice-scrapingprice-scraperweb-crawlerweb-crawlingweb-crawlersamazonamazon-scraperamazon-scraping-libraryscraperdata-miningnode-jsjsscrape-products
  Language:JavaScript 87

• outscraper / outscraper-python

  The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

  google-maps-scrapinggoogle-scrapingscrapinggoogle-maps-apiscraping-pythongoogle-apigoogle-search-apiserp-apigoogle-rating-apigoogle-reviews-apigoogle-serp-apiscraping-websitesgooglegoogle-mapsapiscraper
  Language:Python 84

• fabriziomiano / linkedin-scraper

  Tool to scrape linkedin

  scrapingscraping-pythonlinkedinlinkedin-scraperseleniumbeautifulsoup
  Language:Python 79

• fernandod1 / Instagram-downloader

  Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!

  pythoninstagraminstagram-photosinstagram-scraperinstagram-downloaderinstagram-feedscraperscrapscraping-pythonscrapingscraping-toolscraping-websitescrawlercrawling-python
  Language:Python 74

• sd3v / openinsiderData

  An Openinsider Scraper written in Python. Get the full Database behind the infamous SEC site :)

  insider-tradingpythonscraping-pythonopeninsider
  Language:Python 65

• GEJ1 / web_scraping_freecodecamp

  Curso de web scraping con Python creado por Gustavo Juantorena para freeCodeCamp https://www.freecodecamp.org/espanol/news/aprende-web-scraping-con-python-y-beautiful-soup-en-espanol-curso-desde-cero/

  beatifulsoupespanolfreecodecampweb-scrapingpythonscrapingscraping-python
  Language:Jupyter Notebook 60

• Srivaths-P / UpCode

  Upload accepted solutions from CodeChef and CodeForces to GitHub

  github-apigrequestsmultiprocessingrequestsscraping-pythonjsonpythonpython3seleniumselenium-python
  Language:Python 54

• xtekky / TikDown

  Fast TikTok NO Watermark Video Downloader (username or url)

  pythonrequestsscrapingscraping-pythontiktoktiktok-apitiktok-downloadertiktok-scrapertiktokbot
  Language:Python 44

• TheWebScrapingClub / AI-Cursor-Scraping-Assistant

  A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.

  camoufoxcursor-aicursoraicursorrulesmcppythonscrapingscraping-pythonscrapyscrapy-spiderwebscrapingclaudeclaude-ai
  Language:Python 41

• wael-sudo2 / facebook-page-info-scraper

  Free Facebook pages MetaData Scraping Library - Unlimited Calls

  crawlercrawling-pythonfacebookfacebook-page-informationfacebook-scraperfacebook-scrapingmetadatapythonscraping-pythonseleniumdata-analysisdata-miningcrmleadgenerationmarketingscrapingfacebook-apisfacebook-page-scraper
  Language:Python 41

• mike-gee / webtranspose

  Web scraping API for building AI applications.

  chatbotscrawlingcrawling-pythonpythonscrapingscraping-pythonweb-crawlingweb-scrapingweb-scraping-python
  Language:Python 40

• officialpm / scrape-amazon

  🤩 Python Package for Scraping Amazon Product Reviews ✨

  scrapeamazonbeautifulsoupbeautifulsoup4reviewsproductamazon-reviewsdataamazon-scraping-libraryscraping-dataweb-crawlersscraping-webscraping-pythonscraping-apishacktoberfest
  Language:Python 39

• wetcork / raiplay-dl

  The most advanced raiplay.it downloader

  downloaddownloaderpythonpython3raiplayscrapingscraping-pythonstreamingweb-dl
  Language:Python 39

• alessio-ds / BSCWhalesMonitor

  BSC whales monitor and detector written in Python.

  bscbinance-smart-chainbinance-apibscscanetherscancryptomonitorscraping-pythonscraping
  Language:Python 34

• baronet2 / FirstCyclingAPI

  An unofficial Python API wrapper for firstcycling.com

  cyclingsports-datasportssports-analyticssportsanalyticssports-statssports-sciencecyclinganalyticsapiscraperscraping-websitesscrapingscraping-pythontourdefrancepython
  Language:Python 34

• poudelroshan / BOTVID-19

  Messenger Bot that scrapes for COVID-19 data and periodically updates subscribers via Facebook Messages. Created using Python/Flask, MYSQL, HTML, Heroku

  python3python-3flaskwebappseleniummessenger-botmessenger-chatbotcovid-19mysqlmysql-databasescraperscraping-websitesscraping-pythonscraping-botheroku-deploymentheroku-appfacebook-messenger-botfacebook-bot
  Language:Python 33

• adgramigna / job-board-scraper

  Scrapes job listings from popular job boards Greenhouse, Lever, Ashby, and Rippling

  jobsearchscraping-python
  Language:Python 31

• fernandod1 / ProductHunt-scraper

  Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

  pythonpython3python-scriptscraperscrapingscraping-websitesscraping-pythonscraping-toolscraping-botdata-miningdataminingproducthuntproducthunt-usersproducthunt-apiscrapescraped-datascraper-enginecrawlercrawling-sitescrawling
  Language:Python 29

• mattdood / web_scraping_example

  Simple RSS feed reader for HackerNews.

  datascraping-pythonceleryrss-feed-scraper
  Language:Python 29

• omkarcloud / botasaurus-starter

  🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

  beautifulsoupcrawlercrawlingcrawling-frameworkcrawling-pythoncrawling-toolheadlessnode-crawlerpython-crawlerscraperscrapingscraping-frameworkscraping-pythonscraping-toolseleniumweb-crawlerweb-crawlingweb-scraperweb-scrapingwebscraping
  Language:TypeScript 29

• opplieam / FacebookCrawler

  A web crawler designed for Facebook pages has been developed utilizing the Scrapy framework. This tool is effective for systematic data collection and analysis.

  facebook-botscraping-pythonscraping-websitesreverse-engineeringscrapyweb-scrapingsplash
  Language:Python 29

• kevinam99 / Instagram-Like-Bot

  A simple Instagram like bot that likes posts based on the hashtags you provide.

  instagram-botscraping-pythonselenium-pythonkivypython3python
  Language:Python 28

• asepscareer / ycnbc

  Get Real Time News and Market Data From CNBC Using ycnbc !!!

  newsnewsapiscraping-pythonlxmlcnbc-marketpandaspythonmarket-datacnbc-apistock-marketfinancial-datafinancial-marketsmarkets
  Language:Python 27