ai-scraping

There are 9 repositories under ai-scraping topic.

• mendableai / firecrawl

  🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

  aiai-scrapingcrawlerdatahtml-to-markdownllmmarkdownragscraperscrapingweb-crawlerwebscraping
  Language:TypeScript 20685
• Scrapling

  D4Vinci / Scrapling

  Undetectable, Lightning-Fast, and Adaptive Web Scraping for Python

  aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionhacktoberfestplaywrightpythonpython3scrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
  Language:Python 1752

• itsOwen / CyberScraper-2077

  A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

  ai-scrapinggemini-apillmllm-scraperopenaiscraperweb-scraperwebscraping
  Language:Python 1492

• spider-rs / spider

  A web crawler and scraper for Rust

  ai-scrapingcrawlerheadless-chromeindexerrustspiderweb-crawlerweb-scraperweb-scraping
  Language:Rust 1265

• devflowinc / firecrawl-simple

  ➖ Stripped down, stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features are completely removed. Crawl and convert any website into LLM-ready markdown.

  aiai-scrapingcrawlerdataembeddingshtml-to-markdownllmmarkdownragscraperscrapingsearchweb-crawlerwebscraping
  Language:TypeScript 276

• mendableai / firecrawl-app-examples

  🔥 This repository contains complete application examples, including websites and other projects, developed using Firecrawl.

  aiai-scrapingdataexampleshtml-to-markdownllmmarkdownragscraperstemplatesweb-crawler
  Language:TypeScript 56

• ArchiveBox / abx-dl

  ⬇️ A simple all-in-one CLI tool to download EVERYTHING from a URL (like youtube-dl/yt-dlp, forum-dl, gallery-dl, simpler ArchiveBox). 🎭 Uses headless Chrome to get HTML, JS, CSS, images/video/audio/subtitles, PDFs, screenshots, article text, git repos, and more...

  archiveboxchromecrawlingcurldownloadergallery-dlheadlessinternet-archivingplaywrightpuppeteerscrapingwgetyoutube-dlyt-dlpcli-toolclihttp-clientai-scraping
  Language:JavaScript 52

• spider-rs / web-crawling-guides

  How to guides on web-crawling or scraping

  agentsai-agentsai-scrapingclean-markdowncrawlerhtml-to-markdownllm-webcrawlerscraperweb-scrapingfast-webcrawler
  11

• spider-rs / spider-clients

  Python, Javascript, and Rust libraries for the Spider Cloud API.

  aiai-agentsai-scrapingcrawlerhtml-to-markdownscraperspiderweb-scrapingllm-webcrawlersupabase
  Language:Python 10

• jenslys / skrape-js

  TypeScript/Node.js SDK to easily interact with the skrape.ai API

  aiai-scrapingllmscraperskrapeskrapeai
  Language:TypeScript 0

• nsourlos / AI_tools

  AI tools to enhance productivity and automate web-scraping

  ai-scrapingai-tutorialai-web-scraperaitoolscursorfirecrawlgpt-researchernotebooklmopen-interpreterperplexity-aiscrapegraphaiscreenshot-to-code
  Language:Jupyter Notebook