data-scraping

There are 17 repositories under data-scraping topic.

• web-scraping

  je-suis-tm / web-scraping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  srapingscrapperfuturesfutures-historical-datareuterswall-street-journalbloombergpython-web-scrapernews-websitesfinancial-timesweb-scrapingnews-scraperweb-scrapernewsletterweb-scrapersoptions-datafinancial-datadata-scrapingdata-scraperwallstreetbets
  Language:Python 667

• ScriptSmith / reaper

  Social media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

  scrapingfacebooktwitterreddityoutubepinteresttumblrapisocialmediadata-miningdata-collectiondata-scrapinggui
  Language:Python 355

• anfederico / stockeye

  Keep track of short term stock movement by monitoring breaking news activity

  data-scrapingnews-aggregatorgoogle
  Language:Python 158

• drshahizan / special-topic-data-engineering

  This course presents to the students recent research and industrial issues pertaining to data engineering, database systems and technologies. Various topics of interests that are directly or indirectly affecting or are being influenced by data engineering, database systems and technologies are explored and discussed.

  data-engineeringdatabase-systemdjangonosqldata-science-projectsdata-scraping
  Language:Python 101

• dddat1017 / Scraping-Youtube-Comments

  Scrape comments from any Youtube video

  data-scrapingweb-scrapingpythonselenium
  Language:Python 67

• SwatiModi / e-commerce-web-scraper

  Scraping details of products from ecommerce websites using python

  alibabaamazondata-scrapingecommerceecommerce-web-scraperflipkartpythonselenium
  Language:Python 49

• ohsusannamarie / Instant-Data-Scraper-Chrome-Extension-v0.1.7

  Instant Data Scraper packed Chrome extension v0.1.7 (WITH LinkedIn scraping functionality)

  chrome-extensioninstant-data-scraperdata-scrapinglinkedin-scraper
  Language:JavaScript 47

• deedy / cbse_schools_data

  Cleaned, scraped data of all 20,367 CBSE schools, primarily in India, in 2018. Data scraped from: cbseaff.nic.in/cbse_aff/schdir_Report/userview.aspx

  cbseindiadataopendataschoolseducationexamsindiandata-sciencedata-scraping
  Language:Python 46

• lspahija / torchestrator

  Spin up Tor containers and then proxy HTTP requests via these Tor instances

  scrapingscraping-websitesscraping-tooldata-scrapingscraping-datakotlintordockerproxyproxy-serverproxy-listproxy-checkerproxypoolproxiesproxies-generatorproxies-https-trafficfree-proxiesproxy-scraperprivoxyscrapy
  Language:Kotlin 43

• kb22 / GitHub-User-Insights-using-API

  The project involves using the GitHub API using user authentication to fetch information such as commits and repositories for that specific user and store them as CSV files for data collection and analysis.

  apipythondata-analysisdata-sciencedata-scrapinggithub-api
  Language:Jupyter Notebook 42

• LakshyaKhatri / Bookshelf-Reader-API

  A browsable REST API for recognizing book spines in an image.

  bookrecognizing-book-spinesbookshelf-reader-apiscrappersspinesapirest-apirest-apisrestful-apibookshelfbook-spinesbook-spiderbook-scrapperscrappingdata-scrapingspine-recognitionbook-spine-detectionbook-spinebook-informationspine-detection
  Language:Python 42

• Atharva-Phatak / Analysing-Glassdoor-Jobs

  Data Analysis of Job Postings on Glassdoor.

  glassdoordata-scraping
  Language:Jupyter Notebook 41

• sushil-rgb / AmazonMe

  Introducing the AmazonMe webscraper - a powerful tool for extracting data from Amazon.com using the Requests and Beautifulsoup library in Python. This scraper allows users to easily navigate and extract information from Amazon's website.

  amazonamazon-scraperpythonscraperweb-automationbeautifulsoup4data-scrapingpet-projectdiscord-botdiscord-pyweb-scrapingaiohttpasyncioconcurrent-programmingbotconcurrent-scrapingdatabase-managementmongodbnosql-databaserotating-proxy
  Language:Python 41

• simonw / disaster-data

  Data scraped by https://github.com/simonw/disaster-scrapers

  data-scrapingirma-responsejsongit-scraping
  36

• serpapi / google-search-results-java

  Google Search Results JAVA API via SerpApi

  javajava-apijsonserp-apidata-extractiondata-scrapingserpapiweb-scrapingwebscraping
  Language:Java 31
• Toronto-housing-price-prediction

  slavaspirin / Toronto-housing-price-prediction

  Building Toronto Housing dataset from scratch to predict real estate prices

  machine-learningfeature-engineeringdata-scrapingtorontogeomappredicting-housing-prices
  Language:Jupyter Notebook 29

• zero-to-mastery / Complete-Python-Developer-Manual

  Class notes for Andrei Neagoie's Complete Python Developer course through Zero to Mastery.

  pythonpython3webdevelopmentobject-oriented-programmingscriptingdata-scraping
  Language:Jupyter Notebook 28
• Blox-Fruits-Auto-Farm

  Blox-Fruits-Auto-Farm / Blox-Fruits-Auto-Farm

  Blox Fruits Auto Farm

  authentication-systembloxfruits-autofarm-scriptbookmarkletdata-scrapinglaravelmrzmultilingual-websitespaperwalletphpshopping-cartflask-applicationhotfix-updatehyperledger-fabricjexlpypdf2summarizationstealerblox-fruit-auto-farmblox-fruits-auto-farm
  26

• furkankose / bilkent-scheduler

  Bilkent Scheduler is an open-source tool designed to assist students at Bilkent University in planning their course schedules.

  bilkent-schedulerreactfirebase-functionsfirebase-hostingdata-scraping
  Language:JavaScript 26

• ccubc / ChargeUp

  optimizing locations of electric vehicle charging stations in the city of Toronto

  optimizationawsflaskfoliumpulpdata-scraping
  Language:Jupyter Notebook 25

• rileynwong / spotify-analysis

  Data analysis on my monthly playlists

  machine-learningspotify-analysisaudio-featuresdata-analysislyricstext-analysissentiment-analysisdata-scrapingsupervised-learningsupervised-machine-learningtextnlpnlp-machine-learningnatural-language-processing
  Language:Jupyter Notebook 25

• yuis-ice / jseval

  Evaluate JavaScript on a URL through headless Chrome browser.

  command-lineheadless-browserweb-browserbrowser-automationpupeteerheadless-browserscmdlinecommandline-interfacecli-utilitiesevalevaluatorscrapersdatascrapingscrappingdata-scrapingwebscrappingweb-crawlingscrapperwebsite-scraperweb-scrapping
  Language:JavaScript 24

• lidimayra / raspagem-de-dados-fatec

  :notebook: Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

  scrapingweb-scrapingdata-scrapingpublic-data
  Language:Jupyter Notebook 23

• pinouche / investment_analysis

  automatically fetch value stocks and stocks with unusual option activity (pareto front of iv skew and vrp)

  data-scrapingfinancial-dataoptions-tradingalgorithmic-tradingcryptotrading
  Language:Jupyter Notebook 23

• Smartproxy / eCommerce-Scraping-API

  eCommerce Scraping API code examples for Python, PHP and Node.js

  aliexpress-scraperamazonamazon-scraperamazon-scrapingamazon-scraping-librarydata-miningdata-scrapingecommerce-scrapingecommerce-web-scraperidealoprice-scraperprice-scrapingpython-scrapingscrapescrapingscraping-apiscraping-datascraping-webscraping-websiteswayfair
  Language:PHP 22

• MuhammadAmir5670 / psx-data-reader

  A scraper for downloading Pakistan stock exchange's data into Python Pandas DataFrame.

  pythondata-scrapingstock-exchangepakistan-stock-exchangepakistan-stock-exchage-data
  Language:Python 21

• yashrane / DiscordScraper

  Collects and Analyzes textual data scraped from a Discord chat server

  data-scrapingdata-sciencemachine-learningnatural-language-processingdata-visualizationr
  Language:Python 21

• Smartproxy / Web-Scraping-API

  Web Scraping API code examples for Python, PHP and Node.js

  data-miningdata-scrapingfacebook-scrapinginstagram-scrapernodejs-scrapingphp-scrapingpython-scraperpython-scrapingscraperscrapingscraping-apiscraping-webscraping-websitessocial-media-scrapersocialmediascrapertwitter-scrapingweb-scraperweb-scrapingwebscraperwebscraper-api
  Language:JavaScript 20

• somdeep / Statball

  Statball - Football soccer stats analyser from top 5 european leagues with data obtained by web scraping from Fbref and Statsbomb

  footballfootball-datastatsbombfbrefscrapingdotnetdotnet-coresoccersoccer-analyticsfootball-analyticsdata-scrapingscouting-datacsharptableaudata-vizvisualizationsdata-sciencesoccer-data
  Language:C# 18

• Vyary / exile-profit

  This script engages with APIs to collect data and conduct fundamental computations for identifying the optimal approach to generate profits within the game. It also comes with a preconfigured and continuously updating Google Sheet for seamless utilization.

  data-scrapingpath-of-exilepython
  Language:Python 18

• zohaibbashir / Google-Maps-Scrapper

  This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.

  chromiumdatadata-scrapinggooglegoogle-mapsgoogle-maps-scrapergoogle-maps-scraper-pythongoogle-maps-scrapinggoogle-maps-scrappinggooglemapsmapspythonpython3webscraping
  Language:Python 18

• sallamy2580 / python-web-scrapping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  newsletterweb-scraperweb-scrappingbloombergdata-scrapingfinancial-datafuturesfutures-historical-datanews-scrapernews-websitesreutersscrapersweb-scrapersdata-scraperfinancial-timesoptions-datapython-web-scraperscrapingwall-street-jowallstreetbets
  Language:Python 17

• wingkwong / hk-atm-locator

  :atm: 香港自動櫃員機定位器 :atm: Centralising Automated Teller Machine (ATM) Data in Hong Kong in a well-defined yet standardised format and display in a web portal for public use

  atmhong-kong-atmhk-atm-locatordata-scrapingdata-transformationdata-enrichmentopen-api
  Language:JavaScript 17

• chaitanyarahalkar / Financial-Info-Extractor

  Extract financial information in CSV format for companies compliant to the NSE

  pythonseleniumcsv-parserbeautifulsoupdata-scrapingdata-extractionfinancial-datafinancial-services
  Language:Python 16

• Smartproxy / SERP-Scraping-API

  SERP Scraping API code examples for Python, PHP and Node.js

  baidubingdata-miningdata-scrapinggooglegoogle-scrapergoogle-scrapingpython-scrapingscrapescraperscrapingscraping-apiscraping-datascraping-webserpserp-apiserp-scraperserp-scrapingserpapiyandex
  Language:PHP 15

• easonlai / facebook_post_scraping_and_text_analytics

  This is demo repo to demostrate how to scrape post data from Facebook by Python with library facebook_scraper. And then use Azure Text Analytics to perform sentiment analysis for post text content.

  facebookfacebook-postscraping-pythonfacebook-scraperfacebook-scrapingpythonpython3azuremicrosoft-cognitive-servicestext-analyticsdata-scrapingazure-text-analyticsazure-text-analysismicrosoft-azurepandasseabornkey-phrase-extractionwordcloud
  Language:Jupyter Notebook 14