scrapping

There are 9 repositories under scrapping topic.

• twikit

  d60 / twikit

  Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

  api-wrapperbotclientfreepythonpython3scrapescraperscrapingsearchtwittertwitter-apitwitter-bottwitter-clienttwitter-internal-apitwitter-scrapertwitter-scraper-2023wrapperxx-api
  Language:Python 588

• Xonshiz / comic-dl

  Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.

  automationcomicsdebianmangaphantomjspython-scriptpython-supportscrapingscrapperscrappingwebyoutube-dl
  Language:Python 526
• tinking

  baptisteArno / tinking

  🧶 Extract data from any website without code, just clicks.

  harvestingpuppeteerscrapingscraping-websitesscrapperscrapping
  Language:TypeScript 414
• Python-Roadmap

  hemansnation / Python-Roadmap

  Python Roadmap. Learn Python programming as your first programming language. Python for Absolute Beginners, Non-Tech Professionals, 15+ Projects, 30 Topics, 500+ Practice Questions, with Data Structures & Algorithms

  apiflaskgitgithubherokujupyter-notebookmatplotlibmongodbnumpypandaspostmanpythonscrapping
  Language:Jupyter Notebook 350

• DiegoCaraballo / Email-extractor

  The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

  spyderextractionstractoremail-extractoremailemail-marketingemailsscraperscrapyscrapingscraping-websitesscrapperscrappingscrapy-spiderscraperspython
  Language:Python 173

• tharyckgusmao / Skill-Share-Crawler---DL

  Download Videos Skill Share per ID or per Class

  crawlerdownloadnodejsscrappingskillsharevideos
  Language:JavaScript 128

• clemfromspace / scrapy-cloudflare-middleware

  A Scrapy middleware to bypass the CloudFlare's anti-bot protection

  cloudflarescrapyscrappinganti-bot
  Language:Python 103

• Mati365 / librus-api

  Unofficial advanced node.js Librus scraping API(http://synergia.librus.pl/)

  dzienniklibruslibrus-apilibrus-synergiascrappingszkola
  Language:JavaScript 101

• meetps / mit-ocw-dl

  Download all video lectures from a MIT-OCW course with a single command.

  mit-ocw-dlscrappingvideo-lectures-page
  Language:Python 98

• pwnedshell / Bugs-feed

  Bug's feed is a local hosted portal where you can search for the latest news, videos, CVEs, vulnerabilities...

  pythonscrappinghackingbugbountycvevulnerabilities
  Language:JavaScript 91

• carmolim / olx-monitor

  Script que monitora novos anúncios feitos no OLX, alteração de preços em anúncios existentes e envia notificações para seu Telegram.

  nodejsnotificationolxscrappingtelegram
  Language:JavaScript 71

• msoap / html2data

  Library and cli for extracting data from HTML via CSS selectors

  golangextract-datahomebrewhtmlcss-selectorclilibraryscrappingparser
  Language:Go 66

• lfreneda / statusinvest

  :money_with_wings: statusinvest.com.br stocks info web scraper

  bovespastocksbvmfjavascriptscrapperscrappingibovespa
  Language:JavaScript 57

• shivam5992 / pyscrapper

  :camera: web scrapping in python: multiple libraries -requests, beautifulsoup, mechanize, selenium

  pythonrequestsscrappingselenium
  Language:Python 55

• soulee-dev / fuckvkeypad

  가상키보드(vKeypad) 우회도구

  pythonrequestsrpasecurityseleniumvkeyboardwebautomationkoreancrawlingscrappingvkeypad
  Language:Python 50

• clouedoc / goquotes

  A Go scraper aimed at scrapping quotes.

  golanggoscrapperscrappingcollyquotesh4ck3rz
  Language:Go 49
• pararius-apartment-hunting-dashboard

  isso / pararius-apartment-hunting-dashboard

  🏡 A NodeJS server to scrape Pararius listings and show them in a dashboard 🏘️

  scrappingdashboardmapsamsterdamnetherlandspararius
  Language:CSS 44

• LakshyaKhatri / Bookshelf-Reader-API

  A browsable REST API for recognizing book spines in an image.

  bookrecognizing-book-spinesbookshelf-reader-apiscrappersspinesapirest-apirest-apisrestful-apibookshelfbook-spinesbook-spiderbook-scrapperscrappingdata-scrapingspine-recognitionbook-spine-detectionbook-spinebook-informationspine-detection
  Language:Python 43

• capyvara / brazil-civil-registry-data

  Raw scrapings of ARPEN https://transparencia.registrocivil.org.br/

  covid-19brazildeathsregistryscrappingbrasilmortesregistrocivilcartorioarpencovid
  Language:Shell 39
• fordev

  matheusfelipeog / fordev

  Gere e valide dados randômicos com fordev 🎲

  4devs4devs-api4devs-moduleapidata-generatordata-manipulationdata-validationfake-datafake-data-generatorfordevfourthdevpythonrandom-datascrapping
  Language:Python 34

• bhushankummar / chrome-extension-starter-kit

  Chrome Extension starter Kit with es6 standards

  chrome-extensionchrome-devtoolsscraperscrappingeslinteditorconfigextension-chrome
  Language:JavaScript 32

• kserhii / money-parser

  Price and currency parsing utility

  pythonscrappingmoneycurrency
  Language:Python 26

• yuis-ice / jseval

  Evaluate JavaScript on a URL through headless Chrome browser.

  command-lineheadless-browserweb-browserbrowser-automationpupeteerheadless-browserscmdlinecommandline-interfacecli-utilitiesevalevaluatorscrapersdatascrapingscrappingdata-scrapingwebscrappingweb-crawlingscrapperwebsite-scraperweb-scrapping
  Language:JavaScript 24

• Stvad / google_books_highlight_extractor

  A script to extract highlights and notes from Google Books GDoc to RoamResearch, Markdown, Anki CSV

  pythongoogle-bookshighlightcsvscrappingankireadwiseroamresearch
  Language:Python 23

• its0x08 / duckduckgo

  A simple DuckDuckGo URL scraper.

  duckduckgoddgscraperscrapingscraping-websitesscrapescrappingscrapperurlurlsurl-parsingurl-scraperpythonpython3search
  Language:Python 20

• spaiz / hrscanner

  Websites scanner for X-Recruiting header

  golanggohttp-headersscrappingdns
  Language:Go 19

• davidAlgis / zotero2SemanticScholar

  Send zotero libraby to semantic scholar and add alert on them

  scrappingseleniumsemanticscholarzotero
  Language:Python 18

• NemesisX1 / cine_book_scrapping_api

  A simple api to scrap a popular theater in Africa to get movies and theaters data

  apihacktoberfestpublic-apipuppeteerscrappingtypescript
  Language:TypeScript 18

• tal95shah / OLX_Scraper

  :radio: An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

  scrapypythonpython3olxmongodbpymongoweb-scrapingnosqlweb-crawlingweb-crawler-pythonweb-crawlerweb-scraperscrapernosql-mongodbdatadata-sciencemachine-learningartificial-intelligencescraping-websitesscrapping
  Language:Python 18

• gusty / ScrapeM

  A monadic web scraping library

  computation-expressionsextractfsharpmonadscraperscrapping
  Language:F# 17

• wrobirson / ConsultasDgii

  Provee un API de .NET escrita en el lenguaje C# que permite realizar validaciones de RNC/Cédula en el sitio web de la Dirección General de Impuestos Internos (DGII) de la República Dominicana

  csharpscrapping
  Language:C# 17

• aydinnyunus / ScrapIn

  LinkedIn Scraper

  scraperscrapyscrapingbs4beautifulsoup4beautifulsoupbs4-requestsseleniumselenium-webdriverselenium-pythonpythonpython3python-3linkedinlinkedin-scraperlinkedin-profilescraping-websitesscrapperscrapy-crawlerscrapping
  Language:Python 16

• GreenTreeTeam / WebScraper

  A simple tool made with server-side and client-side for Scraping Websites and downloading them.

  websitescrappingscrapescraperscrape-websites
  Language:TypeScript 15
• AutoMonkey

  MihailCosmin / AutoMonkey

  Python Automation using Mouse and Keyboard, for the masses

  automateautomatedautomationkeyboardmousemouse-eventsmouse-trackingproductivitypyautopyautoguipyautogui-automationpyautogui-hotkeypyautogui-scriptspyautomationpythonrparpa-robotsscrapping
  Language:Python 15

• Shubhamdutta2000 / Web-Scrapping-from-google-with-Scrapy

  Python Scrapy spider that searches Google for a particular keyword and extracts all data from the SERP results. The spider will iterate through all pages returned by the keyword query.

  scrappinggooglegooglesearchscrapy-crawlerscrapperpipenvpython
  Language:Python 14

• whimtrip / jwht-scrapper

  Fully Featured Java Scrapping Framework, highly pluggable and customizable

  javascrapperscrapper-scriptscrappingframeworkhtml-parserhtml-parsing
  Language:Java 14