datascraping

There are 19 repositories under datascraping topic.

• UltimaHoarder / UltimaScraper

  Scrape all the media from an OnlyFans account - Updated regularly

  archivedatascrapingonlyfansscraper
  Language:Python 4184
• Python

  Tanu-N-Prabhu / Python

  This repository helps you learn Python and Machine Learning from scratch.

  pythonjupyter-notebookpandas-dataframenumpypython3python-3numpy-arraysdatadatascrapingdataanalysisgoogle-colabgoogle-colab-notebookmachine-learningpredictiondata-analysisdata-visualizationmachine-learning-algorithms
  Language:Jupyter Notebook 1791
• fansly-downloader

  Avnsx / fansly-downloader

  Easy to use fansly.com content downloading tool. Written in python, but ships as a standalone Executable App for Windows too. Enjoy your Fansly content offline anytime, anywhere in the highest possible content resolution! Fully customizable to download in bulk or single: photos, videos & audio from timeline, messages, collection & specific posts 👍

  cross-platformdatabasedatascrapingdownloaderfanslyfansly-downloadfansly-downloaderfansly-scraperguiimage-downloadlinuxmacosopen-sourceportablepythonredditscrapervideovideo-downloadwindows
  Language:Python 1358

• datawhores / OF-Scraper

  A completely revamped and redesigned fork, reimagined from scratch based on the original onlyfans-scraper

  datascrapingdownloaderfansiteonlyfansscraping
  Language:Python 899

• benibela / xidel

  Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

  xqueryxmlhtmljsonxpathclicommand-linehttpwebrestcss-selectorwgetcurlhttpiexmlstarletwebscraperwebscrapingscraperdatascrapingdata-processing
  Language:Pascal 818

• scrapfly / scrapfly-scrapers

  Scalable Python web scraping scripts for +40 popular domains

  crawlingpythoncrawlerscrapingweb-scrapingweb-scraping-pythonantibotautomationcaptcha-bypasscrawling-pythondatascrapingproxiespython-scraperscraperscraping-pythonspidertwitter-scraperweb-crawlerwebscraperwebscraping
  Language:Python 743

• sim0n00ps / OF-DRM

  C# console app to download DRM protected videos from Onlyfans accounts

  datascrapingof-dlof-drmonlyfans
  Language:C# 208

• DwarfThief / Raspagem-de-dados-para-iniciantes

  Raspagem de dados para iniciante usando Scrapy e outras libs básicas

  estudopythonscrapyjupyter-notebookopensourceweb-crawlerspyderwebcrawlingraspagem-de-dadosdatascrapinghacktoberfest
  Language:Python 136

• Gertje823 / Vinted-Scraper

  This is a tool to scrape/download images and data from Vinted & Depop using the API and stores the data in a SQLite database.

  databasedatascrapingdepopdownloaderpythonpython3scrapersqlitesqlite3vinted
  Language:Python 113

• jordon31 / OnlySnap

  Scrape content from OnlyFans #onlyfans -- #of-scr -- #onlyfans scrape -- #onlyfans-dl -- OnlyFans content downloader -- #of scrap -- #onlysnap

  onlyfanspythonscraperof-scronlyfans-scronlyfans-scrapescrape-onlyfansarchivedatascrapingonlyfans-dlonlysnap
  Language:Python 96

• castlelemongrab / parlance

  A minimum-dependency ECMAScript client library and CLI tool for Parler – a "free speech" social network that accepts real money to buy "influence" points to boost organic non-advertising content

  data-sciencedatasciencedatascrapingdisinformationes7hatespeechjavascriptlaw-enforcementmisinformationnodenodejsosintparlanceparlersocial-mediasocial-networksspeechtwitter
  Language:JavaScript 70

• kennymkchan / funko-pop-data

  Open-source database of all Funko Pop data.

  funko-popdatabasedatascrapingopensourcecsvjson-datacontributions-welcomescraperpuppeteerfunko-pop-scrapperopen-sourcefunko
  Language:JavaScript 58

• arbuzovv / rusquant

  Official version of rusquant package for R

  rdatasourcefinancetradinginvesting-apifinaminvestingcryptocurrencydividendsipoearningssplitsdata-sciencequantquantitative-financedatasetdatasciencedatascrapingstockspackage
  Language:R 45

• jwillmer / web-scraper-chrome-extension

  Web data extraction tool implemented as chrome extension

  webscraperwebscrapingdatascrapingchrome-extension
  Language:JavaScript 28
• Python-Data-Scraping-IMDb-Movie-site-using-BeautifulSoup-Series-1-

  Reljod / Python-Data-Scraping-IMDb-Movie-site-using-BeautifulSoup-Series-1-

  Data Scraping using Python BeautifulSoup

  beautifulsoupdatasciencedatascrapingpythonwebscraping
  Language:Jupyter Notebook 25

• yuis-ice / jseval

  Evaluate JavaScript on a URL through headless Chrome browser.

  command-lineheadless-browserweb-browserbrowser-automationpupeteerheadless-browserscmdlinecommandline-interfacecli-utilitiesevalevaluatorscrapersdatascrapingscrappingdata-scrapingwebscrappingweb-crawlingscrapperwebsite-scraperweb-scrapping
  Language:JavaScript 25

• Agenty / scrapingai

  Build web scraping agents using AI to auto-extract the data from websites, capture screenshot, generate pdf from URL and web crawling with Agenty

  crawlercrawlingdatascrapingextract-datascrapingwebscraperwebscraping
  Language:TypeScript 21

• agnosto / fansly-recorder

  Record fansly streams live and upload to remote using rclone

  datahoardingdatascrapingrclonerclone-backupfanslyfansly-downloader
  Language:Python 21

• dimitryzub / hotels-scraper-js

  Scrape Airbnb, Booking, Hotels.com from a single JavaScript module. ❗No longer maintained.

  webscrapingairbnbbookingdatadatascrapinghotelshotels-apiplaywrightpuppeteerpuppeteer-extra
  Language:JavaScript 18

• kanishkan91 / Python-DataUpdate-DataProcessor-kbn

  The python module can be used to scrape data and process data from different sources. The python module can output data as either as a dataframe in the country year format or it will output data in excel files This module has primarily been created for processing data for the International Futures (IFs) Project however, it can be used to process data in general. The module can be used to process data from the following sources, 1) World Bank World Development Indicators (WDI) 2) UNESCO Education indicators(UIS) 3) FAO Food Balance Sheets (FAO) 4) IMF Global Finance Statistics (IMF GFS) 5) Health data from the Institute for Health and Metric Evaluation (IHME) 6) Water data from FAO AQUASTAT 7) Energy data from EIA Currently this module can be run as is on Windows. For usage on Macs, the user may have to make changes to the code lines which specify paths.

  datapythondataprocessingconcordanceindexingdatascrapingfaoimfaquastatwdiuiseiaihme
  Language:Python 15

• sahilbhange / Facebook-Data-Extraction

  #DataPipeLine #ETL - Created is a Facebook data extraction utility to extract the publicly available data on Facebook. Used Facebook Graph API and Python to extract the data and loaded the data into the CSV files for further analysis.

  facebook-apifacebook-graph-apifacebookdatascrapingfacebook-data-extract
  Language:Python 13

• easonlai / playstore_reviews_scraping_and_text_analytics

  This is demo repo to demostrate how to scrape apps review data from Google Play Store by Python with library Google-Play-Scraper. And then use Azure Text Analytics to perform sentiment analysis for reviews content (aka comments).

  pythonpython3datascrapingdata-scrapingtext-analyticsgoogle-play-storegoogle-play-store-scrapergoogle-play-store-data-analysisazure-text-analyticsazure-text-analysisazuremicrosoft-azuremicrosoft-cognitive-servicessentiment-analysispandasseaborn
  Language:Jupyter Notebook 11

• ice-wzl / DataReaper

  DataReaper is a powerful Python tool designed to harvest data from publicly accessible HTTP servers. It combines the capabilities of Shodan search with web scraping techniques to efficiently gather information from targeted websites.

  data-visualizationdatasciencedatascrapingosintosint-pythonosint-toolpython3redteamvulnerability
  Language:Python 11
• nba-topshot-scraper

  kennymkchan / nba-topshot-scraper

  Node script that will use Selenium to scrape card information from NBA Topshot including card names, rarity, and lowest cost at the moment. Data is scraped once per day.

  open-sourcescraperopensourcenftnba-topshottopshotnbadatascrapingdatabasejson-datacontributions-welcomenft-databasepuppeteernba-topshot-scrapernodeanalytics
  Language:JavaScript 11

• scrape-do / scrapedo-scrapers

  Web scraping examples with Scrape.do 😎

  antibotcrawlerdatascrapingproxiespythonscraperspiderweb-crawlerweb-scrapingwebscraper
  Language:Python 11

• Data-Horde / ytcc-archive

  archiving community contributions on YouTube: unpublished captions, title and description translations and caption credits

  youtubeherokuclosed-captionscaption-contributionsarchivingdatascrapingcommunity-contributions
  Language:Python 9

• VirginiaTech / pyvt

  A Python API for the VT timetable of classes

  datascrapingpythonvirginia-techwebscraping
  Language:HTML 8

• DeDeDeDer / Personal_Projects

  This holds all my personal data-related project's (Automation, Modelling, Analysis)

  python3datasciencedatascrapinginsurance-claimsactuarial-scienceactuarial-statisticsmodelling-frameworkpredictive-modelingexcelvbaclaims-reservingexploratory-data-analysisfeature-engineering
  Language:Python 7

• lavgen / WikileaksAPI-project

  javasctiptnodejswikileaksdatascrapingmongodbmongooseinformation-overloadsonycountleak-groups
  Language:JavaScript 7

• LynnFernandes23 / Movie-Recommedation-System

  I developed a sophisticated movie recommendation system using Python, leveraging key libraries such as Pandas, NumPy, Scikit-Learn, and Natural Language Toolkit (NLTK). The system utilizes data scraping techniques to gather movie information and employs advanced data visualization techniques for insightful analysis.

  data-visualizationdatascrapingexcelnltk-pythonnumpypandaspyhtonscikit-learn
  Language:Jupyter Notebook 5

• cchrisnguyen / FlightRadar24

  A shell script for scraping FlightRadar24's flight tracking data.

  datascrapingflightradar24
  Language:Shell 4

• dimitryzub / py-google-scholar-organic-cite-to-csv-sqlite

  Scrape historic Google Scholar Organic and Cite results to CSV, MySQL Lite using Python and SerpApi.

  webscrapingserpapipythondatadatascrapingwebscrapergooglescholargooglescraperdataextractiondataminingsqlitecsvdatasetdatascience
  Language:Python 4

• greeshmasunil10 / LottoMaxAnalyserBE

  A tool for analyzing the results of the Canadian Lotto Max lottery

  pythonbeautifulsoupdatascrapingmatplotlibnumpyreactanalysislotto-numberslottomax
  Language:Python 4
• greater-toronto-area-housing-data

  kennymkchan / greater-toronto-area-housing-data

  Data scraped from various sites for housing data around the greater Toronto area (GTA). Scrapes happen daily and data is in both JSON and CSV formats. Free to use for analysis.

  datahousing-pricestoronto-open-datadata-scrapingpublic-datareal-estatedata-miningcsvjsontorontoopen-sourcecontributions-welcomedatascrapinghousing-datahousing-dataset
  4

• LynnFernandes23 / Loksabha-Election-2024-Analysis-Through-Power-BI

  This repository hosts interactive dashboards and detailed data visualizations that provide insights into the 2024 Indian parliamentary elections. Utilizing Power BI, we've analyzed voter demographics, electoral results, constituency-wise trends, and more, offering a comprehensive view of the election dynamics.

  datascrapingdatavisualizationexcelpowerbi
  4

• TheOwaisShaikh / Langchainwebsitescraper

  Extract product details from WooCommerce sites using the langchain web extraction library and OpenAI's GPT models.

  chatgptchatgpt-apidatascrapingecommerce-websiteextractionlangchain-pythonopen-aipythonscraping-pythonscraping-webscraping-websiteswoocommercelangchain-data-scrapinglangchian-web-extractionlangchian-web-scrapingwocommerce-product-scrapingwoocommerce-website-scraping
  Language:Python 4