data-scraping

There are 21 repositories under data-scraping topic.

• web-scraping

  je-suis-tm / web-scraping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  srapingscrapperfuturesfutures-historical-datareuterswall-street-journalbloombergpython-web-scrapernews-websitesfinancial-timesweb-scrapingnews-scraperweb-scrapernewsletterweb-scrapersoptions-datafinancial-datadata-scrapingdata-scraperwallstreetbets
  Language:Python 821

• ScriptSmith / reaper

  Social media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

  scrapingfacebooktwitterreddityoutubepinteresttumblrapisocialmediadata-miningdata-collectiondata-scrapinggui
  Language:Python 382

• Voldrix / onlyfans-dl-2

  OnlyFans content downloader v2

  data-scrapingonlyfanswebscraping
  Language:Python 204

• drshahizan / special-topic-data-engineering

  This course presents to the students recent research and industrial issues pertaining to data engineering, database systems and technologies. Various topics of interests that are directly or indirectly affecting or are being influenced by data engineering, database systems and technologies are explored and discussed.

  data-engineeringdatabase-systemdjangonosqldata-science-projectsdata-scraping
  Language:Python 114

• dddat1017 / Scraping-Youtube-Comments

  Scrape comments from any Youtube video

  data-scrapingpythonseleniumweb-scraping
  Language:Python 70

• zohaibbashir / Google-Maps-Scrapper

  This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.

  chromiumdatadata-scrapinggooglegoogle-mapsgooglemapsmapspythonpython3webscrapinggoogle-maps-scrapergoogle-maps-scraper-pythongoogle-maps-scrapinggoogle-maps-scrapping
  Language:Python 66

• ohsusannamarie / Instant-Data-Scraper-Chrome-Extension-v0.1.7

  Instant Data Scraper packed Chrome extension v0.1.7 (WITH LinkedIn scraping functionality)

  chrome-extensioninstant-data-scraperdata-scrapinglinkedin-scraper
  Language:JavaScript 64

• sushil-rgb / AmazonMe

  Introducing AmazonMe, a Python-based web scraper designed to extract data from amazon.com using the requests and beautifulSoup libraries. It simplifies navigation and makes it easy to gather information from Amazon’s website efficiently.

  amazonamazon-scraperpythonscraperweb-automationbeautifulsoup4data-scrapingdiscord-botdiscord-pyweb-scrapingaiohttpasyncioconcurrent-programmingbotconcurrent-scrapingdatabase-managementmongodbnosql-databaserotating-proxyamazon-scraper-python
  Language:Python 64

• SwatiModi / e-commerce-web-scraper

  Scraping details of products from ecommerce websites using python

  alibabaamazondata-scrapingecommerceecommerce-web-scraperflipkartpythonselenium
  Language:Python 54

• deedy / cbse_schools_data

  Cleaned, scraped data of all 20,367 CBSE schools, primarily in India, in 2018. Data scraped from: cbseaff.nic.in/cbse_aff/schdir_Report/userview.aspx

  cbseindiadataopendataschoolseducationexamsindiandata-sciencedata-scraping
  Language:Python 50

• kb22 / GitHub-User-Insights-using-API

  The project involves using the GitHub API using user authentication to fetch information such as commits and repositories for that specific user and store them as CSV files for data collection and analysis.

  apipythondata-analysisdata-sciencedata-scrapinggithub-api
  Language:Jupyter Notebook 45

• LakshyaKhatri / Bookshelf-Reader-API

  A browsable REST API for recognizing book spines in an image.

  bookrecognizing-book-spinesbookshelf-reader-apiscrappersspinesapirest-apirest-apisrestful-apibookshelfbook-spinesbook-spiderbook-scrapperscrappingdata-scrapingspine-recognitionbook-spine-detectionbook-spinebook-informationspine-detection
  Language:Python 45

• lspahija / torchestrator

  Spin up Tor containers and then proxy HTTP requests via these Tor instances

  scrapingscraping-websitesscraping-tooldata-scrapingscraping-datakotlintordockerproxyproxy-serverproxy-listproxy-checkerproxypoolproxiesproxies-generatorproxies-https-trafficfree-proxiesproxy-scraperprivoxyscrapy
  Language:Kotlin 43

• serpapi / google-search-results-java

  Google Search Results JAVA API via SerpApi

  javajava-apijsonserp-apidata-extractiondata-scrapingserpapiweb-scrapingwebscraping
  Language:Java 43

• Atharva-Phatak / Analysing-Glassdoor-Jobs

  Data Analysis of Job Postings on Glassdoor.

  glassdoordata-scraping
  Language:Jupyter Notebook 41

• MuhammadAmir5670 / psx-data-reader

  A scraper for downloading Pakistan stock exchange's data into Python Pandas DataFrame.

  data-scrapingpakistan-stock-exchage-datapakistan-stock-exchangepythonstock-exchange
  Language:Python 38

• simonw / disaster-data

  Data scraped by https://github.com/simonw/disaster-scrapers

  data-scrapingirma-responsejsongit-scraping
  36

• zero-to-mastery / Complete-Python-Developer-Manual

  Class notes for Andrei Neagoie's Complete Python Developer course through Zero to Mastery.

  pythonpython3webdevelopmentobject-oriented-programmingscriptingdata-scraping
  Language:Jupyter Notebook 35

• furkankose / bilkent-scheduler

  Bilkent Scheduler is an open-source tool designed to assist students at Bilkent University in planning their course schedules.

  bilkent-schedulerreactfirebase-functionsfirebase-hostingdata-scraping
  Language:JavaScript 31

• ccubc / ChargeUp

  optimizing locations of electric vehicle charging stations in the city of Toronto

  optimizationawsflaskfoliumpulpdata-scraping
  Language:Jupyter Notebook 30
• Toronto-housing-price-prediction

  slavaspirin / Toronto-housing-price-prediction

  Building Toronto Housing dataset from scratch to predict real estate prices

  machine-learningfeature-engineeringdata-scrapingtorontogeomappredicting-housing-prices
  Language:Jupyter Notebook 29

• sallamy2580 / python-web-scrapping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  newsletterweb-scraperweb-scrappingbloombergdata-scrapingfinancial-datafuturesfutures-historical-datanews-scrapernews-websitesreutersscrapersweb-scrapersdata-scraperfinancial-timesoptions-datapython-web-scraperscrapingwall-street-jowallstreetbets
  Language:Python 27

• yuis-ice / jseval

  Evaluate JavaScript on a URL through headless Chrome browser.

  command-lineheadless-browserweb-browserbrowser-automationpupeteerheadless-browserscmdlinecommandline-interfacecli-utilitiesevalevaluatorscrapersdatascrapingscrappingdata-scrapingwebscrappingweb-crawlingscrapperwebsite-scraperweb-scrapping
  Language:JavaScript 25

• lidimayra / raspagem-de-dados-fatec

  :notebook: Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

  scrapingweb-scrapingdata-scrapingpublic-data
  Language:Jupyter Notebook 24

• pinouche / investment_analysis

  DCA analysis on the s&p 500

  algorithmic-tradingcryptodata-scrapingfinancial-dataoptions-tradingtrading
  Language:Jupyter Notebook 24

• rileynwong / spotify-analysis

  Data analysis on my monthly playlists

  machine-learningspotify-analysisaudio-featuresdata-analysislyricstext-analysissentiment-analysisdata-scrapingsupervised-learningsupervised-machine-learningtextnlpnlp-machine-learningnatural-language-processing
  Language:Jupyter Notebook 24

• Vyary / exile-profit

  This script engages with APIs to collect data and conduct fundamental computations for identifying the optimal approach to generate profits within the game. It also comes with a preconfigured and continuously updating Google Sheet for seamless utilization.

  data-scrapingpath-of-exilepython
  Language:Python 24

• Smartproxy / Web-Scraping-API

  Web Scraping API code examples for Python, PHP and Node.js

  data-miningdata-scrapingfacebook-scrapinginstagram-scrapernodejs-scrapingphp-scrapingpython-scraperpython-scrapingscraperscrapingscraping-apiscraping-webscraping-websitessocial-media-scrapersocialmediascrapertwitter-scrapingweb-scraperweb-scrapingwebscraperwebscraper-api
  Language:JavaScript 23

• somdeep / Statball

  Statball - Football soccer stats analyser from top 5 european leagues with data obtained by web scraping from Fbref and Statsbomb

  footballfootball-datastatsbombfbrefscrapingdotnetdotnet-coresoccersoccer-analyticsfootball-analyticsdata-scrapingscouting-datacsharptableaudata-vizvisualizationsdata-sciencesoccer-data
  Language:C# 23

• chaitanyarahalkar / Financial-Info-Extractor

  Extract financial information in CSV format for companies compliant to the NSE

  pythonseleniumcsv-parserbeautifulsoupdata-scrapingdata-extractionfinancial-datafinancial-services
  Language:Python 22

• Smartproxy / eCommerce-Scraping-API

  eCommerce Scraping API code examples for Python, PHP and Node.js

  aliexpress-scraperamazonamazon-scraperamazon-scrapingamazon-scraping-librarydata-miningdata-scrapingecommerce-scrapingecommerce-web-scraperidealoprice-scraperprice-scrapingpython-scrapingscrapescrapingscraping-apiscraping-datascraping-webscraping-websiteswayfair
  Language:PHP 22

• yashrane / DiscordScraper

  Collects and Analyzes textual data scraped from a Discord chat server

  data-scrapingdata-sciencemachine-learningnatural-language-processingdata-visualizationr
  Language:Python 21

• bilalahhmedd / ebay-products-scraper

  This is ebay products scrapper to scrap images and metadata details of products from ebay.com

  scrapyscrapy-crawlerscrapy-ebayebay-scraperdata-scrapingpython-scraperpython-scrapy
  Language:Python 20

• fdd4s / matterport-downloader

  Download Skybox panoramic photos of Matterport houses

  360-photoequirectangular-panoramamatterportskyboxdata-scrapingreal-estate
  Language:PHP 19
• Bookshelf-Reader

  LakshyaKhatri / Bookshelf-Reader

  An android application that provides information for books kept in a bookshelf by recognizing their spines from images.

  bookspinesbookshelf-readerandroidbookshelfbook-spinesscrapingdata-scrapingcapturespine-recognitionspinebook-spinebook-informationbook-scanningspine-detectionbook-spiderbook-detectionbook-spine-detection
  Language:Java 18

• Smartproxy / SERP-Scraping-API

  SERP Scraping API code examples for Python, PHP and Node.js

  baidubingdata-miningdata-scrapinggooglegoogle-scrapergoogle-scrapingpython-scrapingscrapescraperscrapingscraping-apiscraping-datascraping-webserpserp-apiserp-scraperserp-scrapingserpapiyandex
  Language:PHP 17