data-scraper

There are 4 repositories under data-scraper topic.

• egbertbouman / youtube-comment-downloader

  Simple script for downloading Youtube comments without using the Youtube API

  data-sciencedata-scraperpythonyoutubeyoutube-comments
  Language:Python 1133
• web-scraping

  je-suis-tm / web-scraping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  srapingscrapperfuturesfutures-historical-datareuterswall-street-journalbloombergpython-web-scrapernews-websitesfinancial-timesweb-scrapingnews-scraperweb-scrapernewsletterweb-scrapersoptions-datafinancial-datadata-scrapingdata-scraperwallstreetbets
  Language:Python 834
• awesome-ai-dev-platform-opensource

  AIxBlock-2023 / awesome-ai-dev-platform-opensource

  An On-Chain Open-Source Platform for Rapid AI Model Productization Using Decentralized Resources with Flexibility and Scalability

  aiai-deployment-and-integrationai-development-toolsannotation-toolblockchain-technologycrowdsourcing-platformsdata-crawlerdata-scraperddpdistributed-computingdistributed-data-parallellabeling-toolmlmlopstraining-datasetn8n-alternativeautomationautomation-workflowoff-the-shelf-datasetai-dataset
  Language:TypeScript 551

• kaymen99 / ai-web-scraper

  AI web scraper built with Crawl4AI for extracting structured leads data from websites.

  ai-agentsai-scrapingcrawl4ailead-generationllmsscraperweb-scraperweb-scrapingdata-scraper
  Language:Python 54

• sallamy2580 / python-web-scrapping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  newsletterweb-scraperweb-scrappingbloombergdata-scrapingfinancial-datafuturesfutures-historical-datanews-scrapernews-websitesreutersscrapersweb-scrapersdata-scraperfinancial-timesoptions-datapython-web-scraperscrapingwall-street-jowallstreetbets
  Language:Python 30

• r7avi / Google-Maps-Data-Scrapper

  Google Maps/Place Business Data Scrapper, Scrape data from any google business listed on google maps. Get Name, Address, Phone Number, Plus Code, Reviews, Longitude and Latitude etc. Scrape Emails from Available Websites and Docker Support for automation.

  google-data-scrapegoogle-maps-data-scrapergoogle-maps-scrapergoogle-maps-scraper-pythongoogle-maps-scrappinggoogle-scrapersgoogle-places-scraperdata-scrapergoogle-mapslongitude-and-latitudepythonpython3
  Language:Python 17

• danielfrees / scrapemed

  ScrapeMed: Data scraping for PubMed Central.

  datadata-sciencedata-scraperembeddingsmedical-datapubmedpubmed-centralpubmed-parserpythonscrapingscraping-pythonscrapemed
  Language:Python 15

• AmbitionlessFr1end / h-manga-downloader

  A script for downloading hentai manga/doujin files and storing them in a folder. It is made with python.

  hentaipythonselenium-pythonselenium-webdriverhentai-downloaderdoujinshinhentainhentai-downloadere-hentaihentai2readpururinnhentai-parserprogrammingprogramming-languagescraperscrapingscraping-websitesdata-scrapingdata-scraperparsing
  Language:Python 10

• h26k2 / node-gui-scraper

  A GUI Based Web-Scraper for scraping e-commerce website using NodeJS

  web-scrapperweb-scrapingpuppeteerdata-scraperdata-scrapersdata-scrapingscraperscrapingscraping-websites
  Language:JavaScript 10
• clippy

  abtsousa / clippy

  A robust file downloader and scraper for NOVA School of Science and Technology's own e-learning platform, CLIP.

  data-scraperdata-scrapingfile-downloaderpandaspythonscraperdownloaderlinuxmacstandaloneterminaltuiwindows
  Language:Python 7

• logan-lauton / nba_webscrape

  web scrapes performed for Kaggle datasets.

  basketballbasketball-analysisbasketball-statisticsbasketball-statsdata-analysisdata-sciencedata-scraperdata-scrapingmachine-learningnbanba-analyticsnba-datanba-statisticsnba-statspythonkagglekaggle-dataset
  Language:Jupyter Notebook 7

• M3MONs / hltv-scraper-api

  An unofficial python scraper and flask api for popular CS2 website hltv.org.

  hltvhltv-apihltv-scrapercounter-strikecs2csgoflask-apihltv-newshltvscrapperpython
  Language:Python 7

• FoamoftheSea / tda_scraper

  A web-scraper which uses selenium webdriver to obtain in-depth data from TD Ameritrade website for use with python.

  data-scraperdata-scrapingfinancequantitative-analysisquantitative-financequantitative-tradingstock-analysisstock-datastock-markettd-ameritradeweb-scraperweb-scraping
  Language:Jupyter Notebook 6

• ranbot-ai / instagram-scraper

  A Nodejs script that scrapes data from instagram profiles.

  aidatadata-scraperinstagraminstagram-scraperjsonranbotscraperweb-scraper
  Language:TypeScript 6

• bitArtisan1 / netDigger

  A .NET 8.0 C# WPF desktop application for web scraping data into structured databases with a modern UI, comprehensive logging and optimized high performance.

  csharpdatadata-scraperdata-scrapingdatabasedesktopdotnetinternetlogginguiurlweb-scraperweb-scrapingweb-scrappingscraperweb-scrapers
  Language:C# 5

• farukalamai / yelp-scraper-scrapy-python

  Yelp Restaurant data scraping using python, scrapy spider

  ai-botdata-extractiondata-miningdata-scraperdata-scrapingpythonpython-scraperscrapyscrapy-crawlerscrapy-spiderweb-scraperweb-scrapingweb-scraping-pythonweb-scraping-softwareyelpyelp-apiyelp-restaurantsyelp-resturant-data-scrapingyelp-scraper
  Language:Python 4

• JasonG7234 / NBA-Draft-Model

  This is a Python college basketball data scraper + draft model.

  college-basketball-prospectsdata-scrapernba-draft-model
  Language:Python 4

• logan-lauton / nfl_webscrape

  web scrape performed for Kaggle dataset.

  data-sciencedata-scraperdata-scrapingdatasetfootballfootball-analyticsfootball-datakagglekaggle-datasetmachine-learningnflnfl-playersnflstatsprojectspythonsportssports-analyticssports-datasports-stats
  Language:Jupyter Notebook 4

• Police-Data-Accessibility-Project / docs

  Documentation for the Police Data Accessibility Project.

  police-datadata-scraperpdapdocumentation
  4

• ShahrozAtiq / Hevy-Exercises-Data-Scraper

  This bot automates the process of logging into the Hevy website using your email and password, navigating to the exercises section, and extracting detailed data for each exercise. The extracted attributes include: Name Equipment Primary Muscle Secondary Muscle Source Source Type Thumbnail The data is systematically stored in a CSV file

  data-scraperdata-scrapingdatasetexercisespythonseleniumselenium-webdriverworkout-exercisesexercises-datasethevy-scraper
  Language:Python 4

• ortanaV2 / Data-Scraper

  A data-scraper that makes it possible to filter out the most important information from huge amounts of text based data.

  datadata-scraperfilefile-scraperscrapersearchsearching
  Language:Python 3

• iamdulanga / iqair-data-scraper

  Air quality data scraping tool for get real-time data from https://www.iqair.com/ in Sri Lanka by FECTSL

  air-qualityair-quality-datadata-scraperiqairpurple-air
  Language:HTML 2

• SabujXi / Python-Scraper-and-Data-Analysts-Admin-Panel-in-Django

  A data scraper from texas govt site and a helping web app for managing, reviewing and editing the data

  analystdatadata-analysisdata-entrydata-scraperdjangodjango-applicationpythonpython-scraperreal-estateregexscrapertexas
  Language:Python 2

• adil6572 / qbcc-local-contractor-scraper

  This Python-based web scraper automates the collection of contractor details from the Queensland Building and Construction Commission's (QBCC) website, simplifying data gathering for analysis and database creation in the construction industry.

  beautifulsoup4data-scraperdata-scrapingleadgenerationplaywright-pythonpython-scraperweb-scraper
  Language:Python 1

• DanMesh / ParkrunData

  Data scraper for Parkrun results

  parkrundata-scraperrunningsport
  Language:Java 1

• encoreshao / puppeteer-typescript-starter

  The most basic Puppeteer TypeScript starter

  data-scraperjsonnodemonpuppeteerstructuretypescript
  1

• JDoe0222 / Fake_Job_Scraper

  This is a Repository to show an example of using the BeautifulSoup4 Tool to automate a job search

  automation-testdata-scraperjob-search
  Language:Python 1

• lucifermorningstar1305 / scrappy

  Web Scrapper for any website

  webscrapingpython3data-scrapingdata-scraperdata-scrapping
  Language:Python 1

• luka-j / UpisScraper

  Crawls upin.mpn.gov.rs for data.

  javadata-scraperdata-miningpublic-data
  Language:Java 1

• mertaybat / boardgamegeek-downloader

  Boardgamegeek data downloader

  boardgamegeekdatajsonboardgamegeek-datasetboardgamedata-scraper
  Language:TypeScript 1

• ohiTuna / DATA2000

  tool for scraping state physician DATA2000 waiver counts from SAMHSA

  data2000samhsasuboxonebuprenorphinenaloxoneopiatesgogolangdata-scrapingdata-scraperscraperdatamethadonesubutexwaivermedicinephysicianstateoverdose
  Language:Go 1

• ProMahmudul / goodreads-book-scraper

  The "GoodReads Book Scraper" is a project developed using Laravel 11 framework. This tool aims to extract and organize data from the popular book review platform, GoodReads. With its intuitive interface and powerful scraping capabilities, users can effortlessly gather information such as book original titles, language, ISBN, and published data.

  data-scraperdata-scraper-frameworkgoodreadsgoodreads-apigoodreads-datagoodreads-parserlaravel-scrapperscrapergoodreads-book-scrapergoodreads-scraper
  Language:PHP 1

• ranbot-ai / web-scraper

  A NodeJS script that scrapes metadata from public websites | 2025

  data-scraperencorejsonranbotscraperweb-scraperwebpages
  Language:TypeScript 1

• johnisanerd / Apify-Startup-Investors-Data-Scraper

  Scrape and access thousands of venture investors and family offices.

  data-scraperinvestor-directorystartup-directoriesstartup-directorystartup-internshipsstartup-investingvc-directoryventure-capitalventure-capital-directoryventure-capital-investments
  Language:Python

• ParmodKumar28 / Data-Scraper-Extension

  A Chrome extension built with React.js that allows users to scrape data from any webpage by selecting elements, view random advice fetched from a free API, and export the scraped data into a CSV file.

  chrome-extensiondata-scraperhtml-css-javascriptreactjsvite
  Language:JavaScript

• prashantkoirala465 / TMDB-Movie-Data-Scrapper

  This project is a robust data scrapper that collects comprehensive information about movies from The Movie Database (TMDB) API. The dataset spans from the earliest recorded films (1874) up to recent releases (2024), providing a rich historical record of cinema evolution, production details, popularity metrics, and financial data.

  data-analyticsdata-cleaningdata-miningdata-sciencedata-scraperdata-visualizationdatasetflaskflask-apimachine-learningmovie-data-analysispostmantmdbtmdb-apitmdb-box-officetmdb-database
  Language:Python