url-scraper

There are 5 repositories under url-scraper topic.

• Smartproxy / Smartproxy

  HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

  proxyproxieshttps-proxyproxy-serverpythondata-collectionproxy-integrationproxy-listpython-scraperscrapingurl-scraperweb-scrapingweb-scraping-apiresidential-proxiesdata-gatheringhttp-proxyip-rotationproxy-rotatorproxy-scrapersocks5-proxy
  Language:C# 1088
• serp-scraper-api-guide

  oxylabs / serp-scraper-api-guide

  A quick-start guide on using SERP Scraper API

  pythonscrapergithub-pythonscraping-pythonwebsite-scrapingurl-scrapergoogle-maps-scrapinggoogle-search-apiscraping-apiserp-apiserp-scraperserp-scrapinggoogle-scraperscrape-google-serpserp-scraping-apiapi-for-google-newsapi-google-news
  539
• etsy-scraper

  oxylabs / etsy-scraper

  Etsy Scraper for collecting prices, titles, descriptions, specifications, reviews, and other product information from etsy.com

  etsyetsy-apietsy-api-v3etsy-csvetsy-seoetsy-toolsetsy-scraperurl-scraperetsy-scraping-apietsy-web-scraping
  Language:Java 111

• r3dxpl0it / Damn-Small-URL-Crawler

  A Minimal Yet Powerful Crawler for Extracting all The Internal/External/Fuzz-able Links from a website

  crawlerurl-extractorurl-crawlerurl-parserurl-scraperurl-parameterspenetration-testing
  Language:Python 32
• ebay-scraper

  oxylabs / ebay-scraper

  eBay Scraper for extracting product titles, descriptions, prices, specifications, customer reviews, and ratings from eBay

  ebayebay-apiebay-scraperpython-web-scraperebay-apisebay-searchebay-searchesurl-scraperscrape-ebay
  Language:Java 31

• its0x08 / duckduckgo

  A simple DuckDuckGo URL scraper.

  duckduckgoddgscraperscrapingscraping-websitesscrapescrappingscrapperurlurlsurl-parsingurl-scraperpythonpython3search
  Language:Python 22

• komodoooo / Arb-tool

  Old toolkit

  directory-fuzzinglookupportscannerurl-scraperssl-checkerwebserver-securitydns-enumerationproxy-generator
  Language:Ruby 10
• web-scraping-php

  oxylabs / web-scraping-php

  A tutorial and code samples of web scraping with PHP

  phpweb-scrapingemail-scraperemail-scraper-with-proxyscreen-scrapingurl-scraperwebsite-crawlerwikipedia-scraper
  Language:PHP 8

• JohnScooby / DuckDuckGo-Scraper

  A Simple Script To Scrape DuckDuckGo Search Results Using Python And Selenium WebDriver.

  pythonseleniumdorkingscraperbing-searchdorkdork-scannerduckduckgoduckduckgo-searchgoogle-dorkswebsite-crawlerscrapingbing-dorkingdorking-tooldorkscannerurl-scraper
  Language:Python 7

• krishcdbry / url-scraper

  Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.

  scraptext-parserurl-scrapercheerio
  Language:JavaScript 6

• Gertje823 / Microsoft-stream-url-scraper

  Scrape all video urls of Microsoft Stream

  microsoft-streammicrosoft-stream-downloaderscraperurl-scraperpython3destreamer
  Language:Python 4
• walmart-scraper

  oxylabs / walmart-scraper

  Walmart Scraper for extracting product prices, titles, descriptions, specifications, ratings and reviews from walmart.com

  walmartwalmart-apiwalmart-scraperurl-scraper
  Language:Java 4

• zauberware / url_checker

  Quick lookup for HTTP Status codes of a Sitemap!

  rubyruby-scriptruby-scraperhttp-status-codes404-errorsseotoolsseo-monitorseo-analysisurl-scrapersitemapsitemap-parsernokogiriseo
  Language:Ruby 3

• Cyber-Dioxide / Link-Hawk

  A software for scrapping emails from the entire website. Link Hawk crawls over the entire site and grabs emails from the given domain and its subdomains too.

  crawlercyberdioxideemail-scraperemail-scrapingemailscrappingleads-generationleads-managmentlink-scrappermarketing-automationmoneyurl-scraperwebcrawlerlink-hawk
  2
• amazon-asin-scraper

  oxylabs / amazon-asin-scraper

  Amazon ASIN Scraper for effortless product ASIN extraction.

  amazonamazon-asinamazon-scraperasinamazon-scraper-apiscraper-apiscraper-api-githubamazon-product-scraperamazon-price-trackerurl-scraper
  Language:Python 2
• ecommerce-scraper-api-guide

  oxylabs / ecommerce-scraper-api-guide

  A tutorial on using Oxylabs' E-Commerce Scraper

  e-commerceebay-searchebay-searchesecommerce-apiecommerce-scraperscraper-apiurl-scraperwebsite-crawleremail-scraper
  2

• harindujayakody / fetches-all-links

  Here's a simple Python program that fetches all links from a given webpage and saves the links in a text file.

  fetch-linkspythonpython-3python3url-scraper
  Language:Python 1

• kaanbasal / url-scraper

  Fetch all links recursively from given url

  url-scrapercrawlerpython3asyncio
  Language:Python 1

• Mark10111 / Amz-scrape

  This Amazon Web Scraper script is designed to extract product information from Amazon.com search pages. It retrieves data such as product titles, prices, ASIN, ratings, and review counts.

  automationcybersecurityprojectpythonpython3scraperurl-scraperweb-scraperweb-scraping
  Language:Python 1
• amazon-review-scraper

  oxylabs / amazon-review-scraper

  Amazon Review Scraper for hassle-free review data extraction, including authors, titles, descriptions, ratings, dates, and more.

  amazonamazon-apiamazon-review-scraperamazon-reviewsamazon-scraperamazon-scrapingamazon-product-scraperamazon-scraping-libraryamazon-web-scraperurl-scraper
  Language:Java 1
• how-to-build-web-scraper

  oxylabs / how-to-build-web-scraper

  Step by step guide to building a web scraper with Python

  github-pythonparserpythonpython-web-scraperpython-web-scrapingpython3web-scrapingbuild-a-web-scraperurl-scraper
  Language:JavaScript 1
• scrape-images-from-website

  oxylabs / scrape-images-from-website

  Scrape Images From a Website with Python

  image-scraperscrape-imagespyhton-scraperpython-image-scraperscraper-image-from-websitegithub-pythonurl-scraper
  Language:Python 1
• wayfair-scraper

  oxylabs / wayfair-scraper

  Wayfair scraper for extracting precise titles, prices, reviews, and descriptions from wayfair.com

  wayfairwayfair-scraperurl-scraper
  Language:Java 1

• AIessa / Crawling-Twitter-and-Webscraping

  This repository contains scripts to crawl Twitter for user profiles and scrape web-data using Python libraries.

  tweepy-apitwitter-apicrawling-pythonwebscrapingurl-scraperpython
  Language:Python 0