scrapper

There are 6 repositories under scrapper topic.

• web-scraping

  je-suis-tm / web-scraping

  Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

  srapingscrapperfuturesfutures-historical-datareuterswall-street-journalbloombergpython-web-scrapernews-websitesfinancial-timesweb-scrapingnews-scraperweb-scrapernewsletterweb-scrapersoptions-datafinancial-datadata-scrapingdata-scraperwallstreetbets
  Language:Python 711

• markowanga / stweet

  Advanced python library to scrap Twitter (tweets, users) from unofficial API

  twitterapipythontweetsunofficialsearchcrawlscraperscrapperscrapetwintscrap-tweettwitter-apiuseruserssearchrunnertweetscrap
  Language:Python 578

• Xonshiz / comic-dl

  Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.

  automationcomicsdebianmangaphantomjspython-scriptpython-supportscrapingscrapperscrappingwebyoutube-dl
  Language:Python 537

• iw4p / proxy-scraper

  scrape proxies from more than 5 different sources and check which ones are still alive

  scrappergrabbercheckerproxy-listproxy-checkerproxyscrapersocksproxiesproxy-scrapersocks4proxycheckersocks5socks5-proxyhttps-proxyhttpsocks-proxy
  Language:Python 479
• tinking

  baptisteArno / tinking

  🧶 Extract data from any website without code, just clicks.

  harvestingpuppeteerscrapingscraping-websitesscrapperscrapping
  Language:TypeScript 417

• champmq / TheScrapper

  Scrape emails, phone numbers and social media accounts from a website.

  contactcontact-finderosintosint-toolpythonpython3scrapingscrappersocialmediatool
  Language:Python 223

• rodrigogs / xvideos

  xvideos API library

  xvideospornapilibrarynpmnodejscrawlerscrapper
  Language:JavaScript 209

• keethesh / UdemyCourseGrabber

  Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!

  udemyseleniumselenium-pythonscraperscrapinglxmlscrapperscrapper-script
  Language:Python 183

• DiegoCaraballo / Email-extractor

  The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

  spyderextractionstractoremail-extractoremailemail-marketingemailsscraperscrapyscrapingscraping-websitesscrapperscrappingscrapy-spiderscraperspython
  Language:Python 177

• CuriousLearner / GeeksForGeeksScrapper

  Scrapes g4g and creates PDF

  scrapperpdfgeeksforgeekswebscrapingwebscraperhacktoberfest
  Language:Python 145

• Bishalsarang / Leetcode-Questions-Scraper

  Scrape Algorithm Questions from leetcode and generate html and epub file

  scrapperleetcodeleetcode-downloaderleetcode-offlineleetcode-epubleetcode-pdfleetcode-questionsleetcode-scrapperleetcode-downloadleetcode-problems-solutionsleetcode-savescraping-websitesscraperscrapingdownload-leetcode
  Language:Python 139

• voliveirajr / seleniumcrawler

  An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

  seleniumselenium-webdriverscraperscrapingscraping-websitesscrapperasp-netpythonscrapywebcrawlerwebcrawling
  Language:Python 127

• Joel2B / XVideos-PornHub-RedTube-API

  This script scrapes the HTML from different web pages to get the information from the video (XVideos, PornHub, RedTube) and you can use it in your own video player.

  crawlerphppornhubpornhub-apipornhub-downloaderpornhub-scraperredtubescrapperscrapping-phpvideoxvideos
  Language:PHP 125

• fernandod1 / Instagram-to-discord

  Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

  instagram-scraperinstagraminstagram-photosinstagram-downloaderdiscorddiscord-botdiscordbotdiscordappinstagram-botmonitoring-scriptsmonitormonitorsscraperscrapingscrapperscraping-websitesscraping-pythonpythonpython3webhook-discord
  Language:Python 124

• SpEcHiDe / MyTelegramOrgRoBot

  authmyscrappertelegram
  Language:Python 108

• badoux / goscraper

  Golang pkg to quickly return a preview of a webpage (title/description/images)

  gogolangscraperscrapepreviewwebpagecrawlerenginescrapperimagedescriptiontitleiconpageurlwebsite
  Language:Go 105
• Proxy-List-Scrapper

  narkhedesam / Proxy-List-Scrapper

  Proxy List Scrapper

  proxyfreeproxyfreeproxylistproxyscrapeproxiesscrapperfree-proxiesweb-scrapperdata-miningcrawlercrawlersweb-scrappingrotating-proxyproxy-listproxypoolhttphttpssockssocks4socks5
  Language:Python 96

• lscofield / node-urlresolver-api

  A complete nodejs API to extract streaming sites video direct URLs

  clipwatchingcloudvideodoodstreamgounlimitedjawcloudjetloadkodi-addonmixdropnodejsokrupowvideoscrapperstreamplaysupervideoupstreamuptostreamuqloadvidloxvidozavidtodo
  Language:JavaScript 86

• akasakaid / TiktokDownloader

  Source code of tiktok video downloader without watermark based on telegram

  scrappertelegramtelegram-bottiktoktiktok-download-without-watermarktiktok-downloadertiktok-scrapper
  Language:Python 73

• MLArtist / WebScraper

  Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.

  crawling-pythoncrawlerscraperscrapping-pythonscrapingscrapperwebsite-scraperwebsite-crawlerrobots-txtuser-agentiprotationbeautifulsoupbeautifulsoup4beautiful-soup
  Language:Python 61

• lfreneda / statusinvest

  :money_with_wings: statusinvest.com.br stocks info web scraper

  bovespastocksbvmfjavascriptscrapperscrappingibovespa
  Language:JavaScript 59

• 5hirish / tweet_scrapper

  Scrape the Twitter frontend API without any authentication and restriction.

  python3twitterscrapperlxml
  Language:Python 57

• filipemeneses / fatec-api

  API for SIGA from Centro Paula Souza made for students to create things based in their profile data

  fatecapiscrappersigaprofile-datafatec-api
  Language:TypeScript 56
• goClone

  shurco / goClone

  🌱 goClone - clone websites in seconds

  clonercloningcrawlergogolangscrapingwebsite-clonerwebsite-scraperwp2staticgoclonescrapperhacktoberfest
  Language:Go 55

• clouedoc / goquotes

  A Go scraper aimed at scrapping quotes.

  golanggoscrapperscrappingcollyquotesh4ck3rz
  Language:Go 49

• DreamCobbler / fiction-dl

  A content downloader, capable of retrieving works of (fan)fiction from the web and saving them in a few common file formats.

  fictionfanfictionscrapperformatterdownloaderfiction-dldownloading-storiesebookebook-downloaderepubmobipdfodtscraper
  Language:Python 45

• nicolaspascual / mercadona-scrapper

  Scrapper for the web page of mercadona. It downloads all current products into a csv file

  scrappermercadonapython3selenium
  Language:Python 44

• ZKAW / website-cloner

  Basic website cloner written in Python

  pythonclonerwebsitewebsite-clonewebsite-clonerpython3beautifulsoupbeautifulsoup4scrapscrapperextractorhtmlhtml5downloaddownloadertortor-networkpentestingpentest-tool
  Language:Python 44

• endenwer / tradingview-ws

  TradingView data fetcher through websockets.

  tradingviewtradingwebsocketscrapper
  Language:TypeScript 42

• Potelo / laravel-block-bots

  Block crawlers and high traffic users on your site by IP using Redis

  crawlerslaravelbotsscrapper
  Language:PHP 42

• kant2002 / ncrawler

  Web Crawler written in C#

  crawlerscrapper
  Language:C# 41
• getter

  kastaid / getter

  Get and put users (scraping) to the target group/channel efficiently, correctly and safety.

  telegramtelethonpython3termuxlinuxscrapingpythontelegram-bottelethon-userbotbotuserbottelethon-baseduserbot-telegramuserbotstelegram-botstelegram-userbotscrappergroup-managementpluggablebots
  Language:Python 41

• Romi666 / komikku-api

  REST API Manga bahasa indonesia

  apirest-apiscrappergo
  Language:Go 41

• maforget / Bedetheque-Scrapper-2

  Script that scapes data from Bedetheque.com for ComicRack

  comicrackbedethequescrapper
  Language:Python 40

• satyawikananda / waifu-generator

  Let's pick up your favorite waifu just from the API

  animewaifunodejsscrappervercelserverless
  Language:JavaScript 39

• anshulrgoyal / imdb-scrapper

  This is small project it provide many good api for search ,scrapping ,trending

  imdbscrappernodejavascript
  Language:JavaScript 38