scraper-engine

There are 2 repositories under scraper-engine topic.

• fredwu / crawler

  A high performance web crawler / scraper in Elixir.

  crawlerelixirfilesofflinescraperscraper-enginespider
  Language:Elixir 917

• metatube-community / metatube-sdk-go

  MetaTube SDK/Web Server in Golang

  cloudnativeembygingo-collygormjellyfinpigo-face-detectionpostgresqlscraper-enginesqlite
  Language:Go 557

• TufayelLUS / LinkedIn-Scraper

  A LinkedIn Scraper to scrape up to 10k LinkedIn profiles from company profile links and save their e-mail addresses if available!

  linkedinlinkedin-scraperlinkedin-botleadsemail-scraperscraper-enginescraperemail-marketingdigital-marketingcrawlerscrape-emailsscrape-email
  Language:Python 169

• santhoshse7en / news-fetch

  A Python Package which helps to scrape all news details from any news websites

  newspaper3kgoogle-search-using-pythonnewsscraperscraper-enginenews-detailspythonnews-websitefelixextractskeywordlucasnews-please
  Language:Python 168

• obetomuniz / tatooine

  A powerful scraper for JavaScript Developers.

  nodejshtmlxmljsonspascraperscraper-enginejavascripttypescript
  Language:TypeScript 88

• warifp / Shopee-Scrape

  Shopee Scrape is a tool that functions to collect data - the data needed, such as finding data from photos, prices, names, store locations and others.

  shopeeshopee-apiscraperscrapescraped-datascraper-enginescrape-websitesscrape-imagesmarketplaceindonesiaphpphp-librarycurlcurl-librarycurlphpcurl-functions
  Language:PHP 77

• labteral / bluebird

  Unofficial Python client for Twitter

  scrapingcrawlingsocial-mediaextract-datacrawlerscraperscraper-enginetwittertwitter-apitwitter-bottwitter-clienttwitter-streaming-apitwitter-scrapertwitter-streamtwitter-searchtwitter-scrapingtweets
  Language:Python 43

• flulemon / sneakpeek

  Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

  crawlercrawler-pythoncrawlerscrawlingcrawling-enginecrawling-frameworkpythonpython3scraperscraper-apiscraper-enginescrapersscrapingscraping-frameworkvuewebsite-crawler
  Language:Python 36

• fernandod1 / ProductHunt-scraper

  Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

  pythonpython3python-scriptscraperscrapingscraping-websitesscraping-pythonscraping-toolscraping-botdata-miningdataminingproducthuntproducthunt-usersproducthunt-apiscrapescraped-datascraper-enginecrawlercrawling-sitescrawling
  Language:Python 18

• MrSentex / 0day.today-API

  Unofficial API for 0day.today database | Supported languages: Python and PHP

  inj3ct0rapisearch-apiexploitsexploit-databasescraper-enginepythonphpapi-0day-todaysearch-exploits
  Language:PHP 13

• lukluk / scraper-engine

  Async Scraper Framework Based Nodejs

  scrapingscraper-engine
  Language:JavaScript 11

• fozbek / scrawler

  Simple, schema based scraping tool

  scrawlerscraperscraper-enginescraper-api
  Language:PHP 9

• 1x-eng / chart_data_extractor

  This webservice will help scrape data out of chart(s) presented on any given website. (At this moment, I only support scrape from HighCharts and AmCharts. Other libraries, maybe next time).

  scraperscraper-enginepython-scraperpython-chart-scraperchart-data-extractorchart-datachartshighchartsamchartsamcharts-js-chartshighcharts-jsbeautifulsouppython-web-scraperhacktoberfesthacktoberfest2018
  Language:Python 8

• Vitcesar / plugin.video.anitube

  Add-on para AniTube, famoso site que ofereceu uma gigantesca biblioteca de animes para assistir online e de graça.

  kodipythonanimexbmckodi-addonkodi-pluginkodi-addonsxbmc-video-pluginxbmc-addonxbmc-pluginportugueseportuguesportuguese-languageanitubescraperscrapersscraper-engineanime-scraperstreamingstreaming-video
  Language:Python 8

• fazxid / shopee-product-scraper

  Scrape Data & Images From Marketplace Shopee

  scrapescrapershopeescrape-imageshopee-apiscrape-websitescraper-enginescrape-nodejsscrape-data
  Language:JavaScript 7

• philipperemy / japanese-street-addresses-scraper

  Scraper for Japanese street addresses (住所).

  scraperscraper-enginedataset-generationdataset-creationdata
  Language:Python 7

• animeshkundu / pyscrape

  Lightweight headless web scraper with javascript rendering in python with http api

  scraperscraper-enginepythonscrapejsheadlessjavascriptscrapy
  Language:Python 4

• techieforfun / web-scraper

  :microscope: Scraper for the web

  phplaravelscraper-engine
  Language:PHP 4

• MuhammadAhmed-0 / search-scraper

  This Python-based tool scrapes Google search results and presents the top 10 results along with their URLs. Useful for SEO optimization and content writing.

  googlegoogle-search-scrapergoogle-search-using-pythongoogle-serp-scrapergoogle-serp-scrapingscraperscraper-enginescraper-pythonsearchserp
  Language:Python 3

• gothrek / GotScraper

  Video Game Scraper. Programma che recupera info da database online per arcade games.

  scraper-enginearcademame
  Language:Visual Basic 2

• kingzbauer / scraperlang

  A DSL aimed at making writing web scrapers/crawlers a breeze

  scraperscraper-enginecrawlercrawler-enginegolang
  Language:Go 2

• dasileker / scraper_project

  This project is a capstone project for the Microverse program Ruby, it's a scraper code that searches any torrent name you input.

  rubyscraper-engine
  Language:Ruby 1

• DaveSimoes / Web-Scraping

  The goal of this project is to provide a basic structure for web scraping HTML pages and collecting specific data. The main script (main.py) initializes a WebScraper object and calls the scrape() method to collect data from a specific URL.

  pythonwebscrapingscrapercollectcollect-datehtmlwebscraper-websitescraper-enginescraper-pythonscraper-web
  Language:Python 1

• RBEGamer / OP1.FUN-SCRAPER

  Scrapes all Packages from the Op1.fun Site

  scraper-enginescraperop1opzpackagesamples
  Language:Python 1

• soulomoon / HotelScraper

  A very slow hotel scraper for airbnb, booking, using selenium, beautifulsoup4

  seleniumscraperairbnbbookingwebscrapingwebscraperscraper-engine
  Language:Python 1

• TufayelLUS / LinkedIn-CV-Downloader

  A Python based GUI automation software for downloading bulk LinkedIn CV / LinkedIn Resume from a list of profile links

  crawlerdigital-marketingemail-marketingemail-scraperleadslinkedin-botlinkedin-scraperscrape-emailsscrape-websitesscraperscraper-enginelinkedin-cvlinkedin-cv-downloaderlinkedin-downloadlinkedin-downloaderlinkedin-resumelinkedin-resume-downloader
  Language:Python 1

• vinayakmp007 / Car-folio-scraper

  Scrapes car details from carfolio.com to a xml file

  scraper-enginecar-folio-scraperxmlscraper
  Language:Python 1

• Machine-Learning-Labs / GathererImageGatherer

  Builds a database of Magic the Gathering card images with card name, set, and perceptual hash of artwork.

  pythonmagic-the-gatheringdata-miningscraper-engineimages
  Language:Python 0

• SebastianParzych / Covid-scraper

  Extracting a covid datasets from A charts from the www.worldometers.info.

  beautifulsoupcovid-19scraper-enginedatabase
  Language:Python 0

• vreddi / DotaScraper

  🚽 Scraper for Dota

  dota2scraper-enginedotabuffscrapervisual-studiodota
  Language:C# 0

• breakpointninja / spiderjuice

  scraper-engine
  Language:Python

• seralexger / filmaffinity-scraper

  Unofficial class for scrap and recollect info from Filmmaffinity

  datasetmovie-databasefilm-ratingsdataset-generationdataset-creationdataset-descriptionscrapingscraper-engine
  Language:Python