data-gathering

There are 3 repositories under data-gathering topic.

• news-please

  fhamborg / news-please

  news-please - an integrated web crawler and information extractor for news that just works

  news-crawlernews-extractorcrawlerextractornewsnews-websiteselasticsearchjsonpythonnlpdata-gatheringnews-archivenews-articlescommoncrawlextract-articlesextract-informationnews-scraperccnewscc-newsroberta
  Language:Python 2310
• cacti

  Cacti / cacti

  Cacti ™

  cacticacti-communitycacti-frameworkdata-gatheringgraphgraph-templatingnetwork-discoveryphprrd-filesrrdtool
  Language:PHP 1754

• Smartproxy / Smartproxy

  HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

  proxyproxieshttps-proxyproxy-serverpythondata-collectionproxy-integrationproxy-listpython-scraperscrapingurl-scraperweb-scrapingweb-scraping-apiresidential-proxiesdata-gatheringhttp-proxyip-rotationproxy-rotatorproxy-scrapersocks5-proxy
  Language:Java 1120
• H4X-Tools

  vil / H4X-Tools

  Open source toolkit for scraping, OSINT and more.

  pythonpython-scriptpython3toolshackinglinuxh4x-toolshacking-toolhacktoolsip-scannerphone-numberwebhook-spammerdata-gatheringosintwebscrapingwebsearchdirbusterport-scanneremail-osintigscraper
  Language:Python 418

• lamthuyvo / social-media-data-scripts

  social-mediasocial-webdata-gatheringpythontwitteryoutubefacebooknicar18nicar-2018
  Language:Python 249

• OSINT-TECHNOLOGIES / dpulse

  DPULSE - Tool for complex approach to domain OSINT

  cybersecuritycybersecurity-educationcybersecurity-tooldata-gatheringdomain-analysisgoogle-dorkinginformation-gatheringinformation-securityinfosectoolsintelligenceintelligence-gatheringosintosint-toolosint-toolspentestpentest-toolpentestingweb-scrapingwebscrapingwebsite-scraper
  Language:Python 109

• shadawck / glit

  Retrieve all mails of users related to a git repository, a git user or a git organization

  osintdata-gatheringemailmultithreadedosint-toolrustemail-osintinformation-gathering
  Language:Rust 48

• viralvaghela / Jwiki

  Java tool to get wikipedia data

  javawikipediawikipedia-apijavawikipedajavatoolwikipedia-scraperdata-gathering
  Language:Java 36

• chrislicodes / udacity-data-analyst-nanodegree

  Repository for the projects needed to complete the Data Analyst Nanodegree.

  udacitydatadata-analysisdata-visualizationdatasetdata-cleaningdata-analyticsdata-analyst-nanodegreestatisticsdata-wranglingdata-gatheringweb-crawlingapitext-miningpandasnumpymatplotlibseaborntweepy
  Language:Jupyter Notebook 32

• DialRC / PortalAPI

  Portal Tutorial

  api-clientapi-serverdialogdialog-systemsdata-gatheringportal-tutorial
  Language:Python 11

• lucaromagnoli / dataservice

  Python async data gathering

  asyncdata-gatheringpythonwebcrawlingwebscraping
  Language:Python 11

• BahramJannesar / CafebazzarWebsiteScraper

  Data gathering from https://cafebazaar.ir

  data-sciencedata-gatheringscraperwebscrapingcafebazzaraaron-swartz
  Language:Python 6
• booru-dataset-gatherer

  Corruptex / booru-dataset-gatherer

  A .NET Core 3.1 Console application to gather tags and relevant information from Booru websites for Machine Learning.

  boorucsvdata-gatheringlatent-diffusionmachine-learningdata-miningdatasetnetcorenetcore31stable-diffusion
  Language:C# 5

• dbrennand / twitter-stream-bot-data-gatherer

  An application to watch the Twitter stream and send accounts to the Botometer API for analysis. The results are stored in a SQLite database.

  twitter-apitwitter-bot-detectionbot-detectiontwitter-streaming-apidata-gatheringbotometer
  Language:Python 5

• rdempsey / data-gathering-and-wrangling

  Code and slides for my class: Data Gathering & Wrangling

  data-sciencedata-gatheringdata-wranglingpythonapiweb-spiderwebscrapingwebscraper
  Language:Python 5
• Oil-vs-BigTech-stock-investigation

  sondosaabed / Oil-vs-BigTech-stock-investigation

  💹📈Investigating the oils market prices in addition to the stock market prices between the start of 2001 to the end of 2023. 💰📉

  data-analyst-nanodegreedata-assessmentdata-gatheringdata-sciencepythonwrangling-dataadvanced-data-wranglingapirequests
  Language:Jupyter Notebook 5

• subahanii / COVID19-tracker

  This is for Indian cases and data gathering from Indian governments site.

  covid19covid19-indiacovid19-live-trackercovid19-graphcovid-2019covid19-datacovid-apicovid19-trackercovid-19covid-19-indiadjangodjango-rest-frameworkdjango-projectbs4requests-moduleindian-governmentsindian-casesdata-gatheringwebapppython
  Language:HTML 5

• calista-ai / crowdsourcing-app

  A Web Application to collect data from pairwise image comparisons via crowdsourcing

  web-apppairwise-comparisonsdata-collectiondata-gatheringimage-evaluationdatasetdataset-creationcrowdsourcingmern-stackreactnodeexpressmongodbdockernginxmongo-expresscalistacalista-aiwebsite-aesthetics
  Language:JavaScript 4

• openpilot-community / opc-web

  Seeking Maintainers. The official codebase for the openpilot community info portal.

  autonomous-vehiclescarcarscommaaidata-gatheringdrivingopenpilot-communityself-drivingvehicle
  Language:JavaScript 4

• Shahrukh2016 / Retail_Sales_Prediction

  Revolutionize sales forecasting for Rossmann stores with our high-accuracy XGBoost model, leveraging data analysis, feature engineering, and machine learning to predict sales up to six weeks in advance.

  data-analysisdata-cleaningdata-gatheringdata-preprocessingdecision-treesexploratory-data-analysisfeature-engineeringlightgbmmachine-learningmodel-evaluationmodel-selectionone-hot-encodingoutlier-detectionrandom-forestsregression-analysisregularizationstatistical-analysistransformationvisualizationxgboost
  Language:Jupyter Notebook 4

• speckly / sucorn

  ML/DL dataset collection utilities

  classificationdata-gatheringdeep-learningdiscordstatisticssupervised-learning
  Language:Python 4

• Yara-Alotaibi / Wrangle-and-Analyze-WeRateDogs-Data

  This is Udacity's Data Analyst Nanodegree's 5th project; which is Wrangling and Analyzing WeRateDogs Twitter account.

  data-analysisdata-analyticsdata-analysesdata-visualizationdata-wranglingdata-gatheringdata-assessmentdata-cleaningdata-scrapingudacityudacity-nanodegreeudacity-data-analyst-nanodegreepyhtonpython3twittertwitter-apitweepy
  Language:Jupyter Notebook 4

• darsan-in / Job-Crawler

  The Job Crawler is an integral component of the Job RAID project, designed to automatically scrape and collect data from various job listing websites. This crawler enables Job RAID to aggregate comprehensive job listings, ensuring that users have access to up-to-date and relevant job opportunities.

  automated-job-listingscrawler-integrationdata-extractiondata-gatheringjob-aggregatorjob-crawlerjob-datajob-data-collectionjob-data-minerjob-listing-crawlerjob-portal-scrapingjob-scrapingjob-scraping-tooljob-search-automationjob-search-enginemulti-site-job-scrapingreal-time-job-datascraping-jobsweb-crawlerweb-scraping
  Language:Python 3

• joedarby / AlcoSensing

  An Android app for mobile sensing research

  androidalcoholdata-gatheringresearchhuman-behavior
  Language:Java 3

• rhart-rup / Foursquare-Venue-Scraper-2023

  Find all venues in a geographic area using the Foursquare API and collect extended Foursquare data on each venue.

  data-gatheringdata-sciencefoursquare-apijupyter-notebookpythonscrapersearchvenues
  Language:Jupyter Notebook 3

• ctroller / rlfantasy

  Custom Rocket League Fantasy League Stats Aggregator

  rocket-leaguefantasy-leaguedata-gatheringjavamaven
  Language:Java 2

• kleinpa / keyboardtime

  Foreground application logger for Windows

  personal-analyticsdata-gatheringusage-dataproductivity
  Language:Python 2

• petermeissner / diffrprojects

  rstatsnlptext-comparisondata-gatheringdata-sciencedata-structure
  Language:R 2

• rafa-br34 / MCSF

  Minecraft Server Finder is a small toolkit which helps in finding Minecraft servers and tracking players using the "sample" parameter.

  data-gatheringdata-harvestingminecraftscanningminecraft-servernetwork-scanningosintosint-toolkit
  Language:Python 2

• wwengm / findar

  Financial Datareader

  financedata-gatheringbacktesting-trading-strategies
  Language:Python 2

• ZawszeBaka / face_recognition

  Face Detection => Data Gathering => Training => Face Recognition

  face-recognitionface-detectiondata-gathering
  Language:Python 2

• 2020-HelloWorld / SolutionToCrowdSourcingData

  This is the project for Karnataka Police Hackathon

  policecriminal-justicesearchingdata-gatheringdata-discoveryanalysissecuritykali-linuxtruecaller-apifacebook-developersinstagram-scrapertwitter-api
  Language:Python 1

• kdvalin / ipmi-exporter

  Exports IPMI sensor information to a CSV

  csvdata-gatheringipmiimpitoolsensors
  Language:Python 1

• shishir-t3 / Dive_Into-PlayStore

  A project involving Web Scraping of App and reviews info from the Google play store and building a recommendation system deployed using Streamlit.

  data-gatheringdata-visualizationdeep-learningexploratory-data-analysismachine-learningpca-analysispython3recommendation-systemstreamlitt-snetensorflow2web-scraping
  Language:Jupyter Notebook 1

• sorrychoe / covid19today

  DEPRECATED: Today's World covid-19 Data Gathering Tool

  covid19-datadata-extractiondata-gatheringr
  Language:R 1

• TelRich / Web_Scrapping_with_BeautifulSoup-and-Wptool

  Web scraping Webometrics and Wlkilpedia using Python (Beautiful soup and Wptools) to make a list of top 100 Universities in nigeria

  api-pythonbeautifulsoup4data-extractiondata-gatheringjson-pythonpandas-dataframerequests-library-pythonweb-scrapingwptools
  Language:HTML 1