python-scraper

There are 11 repositories under python-scraper topic.

• SeleniumBase

  seleniumbase / SeleniumBase

  Python APIs for web automation, testing, and bypassing bot-detection with ease.

  pythonseleniumwebdriverselenium-pythone2e-testingseleniumbasepytest-pluginweb-automationpytestwebkitchromedriveranti-detectionbot-detectioncloudflare-bypassweb-scraping-pythonpython-scraperweb-scrapingtest-automationcdpbehave
  Language:Python 11848
• botasaurus

  omkarcloud / botasaurus

  The All in One Framework to Build Undefeatable Scrapers

  anti-botanti-detectioncloudflare-bypasscloudflare-scrapeanti-detectanti-detect-browserantidetect-browserundetectedundetected-chromedriverbypass-cloudflarepython-web-scraperpython-web-scrapingscraping-frameworkscraping-toolundetectableweb-scraping-pythonbot-detectionscraping-pythonweb-crawlingpython-scraper
  Language:Python 3194

• Decodo / Decodo

  HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

  proxyproxieshttps-proxyproxy-serverpythondata-collectionproxy-integrationproxy-listpython-scraperscrapingurl-scraperweb-scrapingweb-scraping-apiresidential-proxiesdata-gatheringhttp-proxyip-rotationproxy-rotatorproxy-scrapersocks5-proxy
  Language:Java 1146

• scrapfly / scrapfly-scrapers

  Scalable Python web scraping scripts for +40 popular domains

  crawlingpythoncrawlerscrapingweb-scrapingweb-scraping-pythonantibotautomationcaptcha-bypasscrawling-pythondatascrapingproxiespython-scraperscraperscraping-pythonspidertwitter-scraperweb-crawlerwebscraperwebscraping
  Language:Python 747
• instascrape

  chris-greening / instascrape

  Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically

  pythoninstagramwebscrapingdata-mininginstagram-scraperlightweightpython3data-sciencepython-scraperinstagram-databeginner-friendly
  Language:Python 659
• how-to-scrape-google-scholar

  oxylabs / how-to-scrape-google-scholar

  A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

  google-scholargoogle-scholar-scraperpythonpython-scraperscraper-apiweb-scraperweb-scrapinggoogle-scholar-scrappergoogle-search-scraper
  Language:Python 587
• how-to-scrape-amazon-prices

  oxylabs / how-to-scrape-amazon-prices

  A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

  amazonamazon-scraperapipythonpython-scraperscraper-apiweb-scraperweb-scraping
  Language:Python 534

• TheSpeedX / WhatScraper

  A Whatsapp Group Link Scraper And Checker

  whatsapp-botwhatsappscraperpython-scraperspeedxhacktoberfest
  Language:Python 231

• yousefkotp / Movies-and-Series-Scraper

  A console application to scrape a valid watching links for any movie or series with exact season and episode number, you can also download a whole season with one click.

  scraperseriesseries-scrapermovie-scrapperbeautifulsoupbeautifulsoup4moviemoviespythonpython-scraperpython-scrapingpython-scriptscrapingscraping-websites
  Language:Python 138

• thewebscraping / tls-requests

  TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

  tls-clientcloudflare-scraperweb-scraping-apicloudflare-bypasscf-clearanceanti-botanti-bot-detectionanti-bot-pagepython-tls-clientpython-crawlerpython-spiderweb-scraping-pythonpython-web-scraperpython-web-scrapingpython-web-crawlerweb-spiderscraping-pythonweb-crawler-pythonpython-scrapercrawling-python
  Language:Python 108

• Mohamedsaleh14 / Reddit_Scrapper

  Scrape Reddit for marketing pain points using GPT — built to fuel Cronlytic's growth.

  redditreddit-apigptcroncronjob-schedulergpt-4o-minigpt4python-scraperscraperscrappinggpt-4gpt-4ogpt-apigpt-4-apiaiautomationmarketing
  Language:Python 93

• harismuneer / Android-Apps-Downloader

  📱 A utility for downloading Android apps from the Google Play Store and Xiaomi App Store (the Chinese App Store).

  scrapercrawlerandroid-researchpython-scraperxiaomi-appsandroid-application-downloaderscraping-toolcrawling-toolopen-source-projectgoogle-play-store-scraperxiaomi-store-scraperwget-utilitygplaycliresearch-toolandroid-scraperandroid-apps-crawlerandroid-toolgoogle-play-application-downloaderapp-downloaderandroid-market-scraper
  Language:Python 35

• dootss / catbox-scraper

  An extremely fast script for scraping file hosting site catbox.moe for any file extension.

  catboxfile-scraperpython-scraperrandom-imagesscrapercatbox-moe
  Language:Python 26

• Decodo / Web-Scraping-API

  Web Scraping API code examples for Python, PHP and Node.js

  data-miningdata-scrapingfacebook-scrapinginstagram-scrapernodejs-scrapingphp-scrapingpython-scraperpython-scrapingscraperscrapingscraping-apiscraping-webscraping-websitessocial-media-scrapersocialmediascrapertwitter-scrapingweb-scraperweb-scrapingwebscraperwebscraper-api
  Language:JavaScript 25

• bilalahhmedd / ebay-products-scraper

  This is ebay products scrapper to scrap images and metadata details of products from ebay.com

  scrapyscrapy-crawlerscrapy-ebayebay-scraperdata-scrapingpython-scraperpython-scrapy
  Language:Python 22

• SuperKogito / CoinMarketCapScraper

  a small python scraper to scrape historical data from the CoinMarketCap website and convert it to csv files . This is an initial step for a data mining process to develop a predictive model of cryptocurrencies prices.

  scraperscraped-datacoinmarketcapcryptocurrencycsv-filesbig-datacryptocurrencies-pricespython-scrapercoinmarketcap-websitecsv
  Language:CSS 20

• LaskasP / TripAdvisor-Python-Scraper-Restaurants-2021

  An implementation on python for scraping unique stores or scraping a whole geographical area from TripAdvisor

  tripadvisorpython-scraperrestaurants-reviewsreviewsscraperarearegiontripadvisor-scrapertripadvisorreview
  Language:Python 19
• mapScraper

  christivn / mapScraper

  🤖🗺️ Headless browser scraper written in python to extract Places data from Google Maps.

  google-map-apigoogle-mapsgoogle-maps-apigoogle-maps-scrapergoogle-maps-scrapinggoogle-placesleadlead-generationprospectingprospectspython-scraperserapipythongoogle-scraper
  Language:Python 17

• shaheenfazim / url-scraper

  A python script to scrape URL's from major search engines.

  bing-scraperduckduckgo-scrapergoogle-scraperpython-scraperscraperweb-scraperyahoo-scraper
  Language:Python 17

• hhhrrrttt222111 / Selenium_python

  pythonpython-scraperseleniumselenium-webdriverselenium-pythonbeatifulsoupscraping-websitesscraping-pythonwebscrapingrequests-htmlwebscraperpython-requestsbeautifulsoup4beautifulsoup-libraryspiderspycharm-idechromedrivergeckodriverpython-parserhacktoberfest
  Language:Python 15

• 0x4D-5A / e-scraper

  Python script to extract unique email addresses from a list of domains using regular expression.

  email-crawleremail-extractionemail-extractoremail-marketingemail-scraperemail-scrapingpython-scraperscraping-websites
  Language:Python 14

• 1x-eng / chart_data_extractor

  This webservice will help scrape data out of chart(s) presented on any given website. (At this moment, I only support scrape from HighCharts and AmCharts. Other libraries, maybe next time).

  scraperscraper-enginepython-scraperpython-chart-scraperchart-data-extractorchart-datachartshighchartsamchartsamcharts-js-chartshighcharts-jsbeautifulsouppython-web-scraperhacktoberfesthacktoberfest2018
  Language:Python 12

• Genone22 / emails_phones_scraping

  Extract emails and phone numbers from the list of url addresses

  email-scraperemail-scrapingextract-dataparsingpython-scraperscraping
  Language:Python 12

• mzubairtahir / Google-Maps-Scraper

  Scraper that scrapes data from search results of google maps

  business-datadatadata-scrapinggoogle-map-apigoogle-map-scrapergoogle-mapslead-generatorscraperscraping-websitespython-scraperpython-scrapingpython3
  Language:Python 12

• RGProjectX / yts-torrent-scrapper

  A Python Program Based On BeautifulSoup To Search The User's Desired Movies On https://yts.mx And Return Movies And Torrent Info

  yts-torrentyts-apiyts-moviesscraping-websitesbeautifulsouptorrentstorrent-scrapingpython-scrapermovie-scrapperyify-torrentsyify-movies
  Language:Python 11

• lc-at / bni-api

  A Python wrapper for some of BNI's internet banking functionalities.

  internet-bankingscraperpython-scraperlxmlwrapperpython3python
  Language:Python 10

• MSadeghzadehG / Scrap_Learning

  Scrap a website with bypass the captcha

  python-scraperbypasscaptcha
  Language:Jupyter Notebook 10

• sakshamssr / upanime

  UpAnime : Modern Search Engine API for Manga.

  animeapicomicscomics-apimangamanga-apimanga-scraperpythonpython-scraperpython-scriptpython3scraperanime-apirest-apimanga-readermangareadermangadexpython-scriptingweb-scraperweb-scraper-api
  Language:Python 10

• sbmagar13 / raidfscrape

  Code to Scraping some portion of Data from forum(RaidForums[seized by FBI]) with Python SCRAPY spiders bypassing recaptcha and storing to PostgreSQL database.(used scrapper-API as captcha, proxy bypass) SQLAlchemy as ORM for PostgreSQL - Python.

  scraping-websitesscraping-pythonscrapy-crawlerforum-scrapingscrapy-spiderpython-scraperdata-scrapingscrapper-api
  10

• notscar / zalando-scraper

  Zalando coming soon releases scraper sent with discord webhooks

  zalandoscraperzalando-scaperpython-scraper
  Language:Python 9

• oxylabs / scraping-real-estate-data-with-python

  A guide for extracting public data from Redfin using Python and Oxylabs Web Scraper API. From using the API to processing data and saving the result.

  e-commerce-scraperecommerce-apiecommerce-scraperpythonpython-scraperreal-estatescraper-apiweb-scraping
  Language:Python 9

• TheAdnan / hadith-of-the-day

  Firefox WebExtension that displays a different hadith on click

  hadithscraperpython3beautifulsoup4python-scraperwebextensionfirefox-extensionfirefoxfirefox-webextensionfirefox-addon
  Language:HTML 9

• kishanmodi / Torrent-Scrapper-API

  An API to Scrap Magnet/Torrent Links from 1337x, It provides the top seeded result from your query. Scrapping is done using BeautifulSoup4 and Python3. API is hosted on Heroku.

  bs4flaskherokupython-scraperpython3scrapping1337x-apimagnet-linktorrenttorrent-scrapertorrent-search-enginehacktoberfesthacktoberfest-accepted
  Language:Python 8

• luminati-io / web-scraping-with-pydoll

  Use Pydoll to scrape JavaScript-heavy sites, bypass Cloudflare, and scale web scraping with rotating proxies like Bright Data.

  python-scraperweb-scraperweb-scrapingpydoll
  8

• Touexe / LinkTreeScraper

  Scraping Linktree user's information and links

  linktreelinktree-apiscraperpython-scraper
  Language:Python 8

• androiddevnotesyoutube / scrape-twitter-json-to-html-table

  This script shows how to scrape your tweets by date or year using the opensource project Twint and turn it into a beautiful HTML table. No API key is required..

  twitterjsonpythonfilterscriptscrapescraperpython-scriptpython-scraperbeautifulsoup
  Language:Python 7