data-extraction

There are 24 repositories under data-extraction topic.

• firecrawl

  firecrawl / firecrawl

  🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

  aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
  Language:TypeScript 66898

• ScrapeGraphAI / Scrapegraph-ai

  Python scraper based on AI

  scrapingscraping-pythonautomated-scraperllmweb-crawlerweb-scrapingai-scrapingcrawlermarkdownragweb-crawlersai-crawlerai-searchlarge-language-modelweb-data-extractionweb-searchweb-scraperdata-extractionweb-datawebscraping
  Language:Python 21729

• getmaxun / maxun

  ⚡ Easiest no code web data extraction platform • Instantly turn any website into API or spreadsheet ⚡

  automationno-codescraperweb-automationweb-scraperweb-scrapingapibrowserbrowser-automationplaywrightself-hostedrobotic-process-automationrpano-code-web-scraperagentsdata-extractionwebscrapinghacktoberfesthacktoberfest-acceptednocode
  Language:TypeScript 13826
• Scrapling

  D4Vinci / Scrapling

  🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

  aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
  Language:Python 8110

• vi3k6i5 / flashtext

  Extract Keywords from sentence or Replace keywords in sentences.

  data-extractionkeyword-extractionnlpsearch-in-textword2vec
  Language:Python 5681
• contextgem

  shcherbak-ai / contextgem

  ContextGem: Effortless LLM extraction from documents

  aicontract-analysisdata-extractiondocument-intelligencegenerative-ailegaltechllmllm-extractionllm-frameworkllm-pipelinellmsnlpprompt-engineeringtext-analysisunstructured-datadocxdocx2mddocx2txt
  Language:Python 1705

• JonathanLink / PDFLayoutTextStripper

  Converts a pdf file into a text file while keeping the layout of the original pdf. Useful to extract the content from a table in a pdf file for instance. This is a subclass of PDFTextStripper class (from the Apache PDFBox library).

  layouttextjavapdfextractdata-extractionpdfbox
  Language:Java 1595

• brightdata / brightdata-mcp

  A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

  llmmcpmodelcontextprotocolscrapingai-agentsai-integrationsanti-bot-detectionbrowser-automationdata-collectiondata-extractionmcp-serverscraping-toolsstructured-dataweb-crawlingweb-dataweb-scraping
  Language:JavaScript 1557

• hi-primus / optimus

  :truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

  big-data-cleaningbigdatacudfdaskdask-cudfdata-analysisdata-cleanerdata-cleaningdata-cleansingdata-explorationdata-extractiondata-preparationdata-profilingdata-sciencedata-transformationdata-wranglingmachine-learningpysparkspark
  Language:Python 1524

• raznem / parsera

  Lightweight library for scraping web-sites with LLMs

  aiai-scrapingdata-extractionllmopensourceplaywrightpythonscrapingwebscraping
  Language:Python 1235

• saifyxpro / HeadlessX

  A lightweight, self-hosted headless browser automation platform. Designed as an alternative to Browserless, built for speed, privacy, and scalability.

  automationbrowser-automationbrowserlesschromedriverheadlessautomation-apiautomation-platformbrowser-testingchrome-headlessdata-extractionheadless-chromeheadless-serviceplaywrightplaywright-automationpuppeteerscraping-serviceweb-automationweb-scrapingcontainer-automation
  Language:JavaScript 1059
• vnstock

  thinh-vu / vnstock

  A beginner-friendly yet powerful Python toolkit for financial analysis and automation — built to make modern investing accessible to everyone

  stock-marketdata-extractionstock-screenerquantitative-financequantitative-analysisquantitative-trading
  Language:Python 1028
• hacker-news-digest

  polyrabbit / hacker-news-digest

  :newspaper: Let ChatGPT Summarize Hacker News for You

  hacker-newspythondata-extractionhacker-news-readerrssextract-summarieshacker-news-digestspidercrawlermachine-learningnews-aggregatorchatgptchatgpt-apiopenaiopenai-api
  Language:Python 733

• adrienjoly / npm-pdfreader

  🚜 Parse text and tables from PDF files.

  data-extractionpdf-converterparsingjavascripttabular-datapdf-readerparse-tablesrule-based-parsing
  Language:HTML 692

• ScrapeGraphAI / scrapecraft

  🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.

  aiautomationdata-extractiondockerfastapihacktoberfestlanggraphpythonreactscrapegraphaitypescriptweb-scrapingwebscraping
  Language:Python 551

• eclaire-labs / eclaire

  Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.

  aiai-assistantautomationbookmark-managerbookmarksdata-extractiondocument-processingllmlocal-firstnote-takingocron-device-aiopen-sourcepersonal-knowledge-managementprivacyrest-apiself-hostedtask-managementweb-archiving
  Language:TypeScript 487

• a-maliarov / amazoncaptcha

  Pure Python, lightweight, Pillow-based solver for Amazon's text captcha.

  captchacaptcha-solveramazonpython3pillowamazon-captchaamazon-scrapertraining-dataamazoncaptchadata-extraction
  Language:Python 486

• py-pdf / benchmarks

  Benchmarking PDF libraries

  benchmarkdata-extractionmupdfpdfpoppler-utilspypdf2text-extraction
  Language:Python 315

• jpjacobpadilla / Stealth-Requests

  Undetected web-scraping & seamless HTML parsing in Python!

  pythonhttp-clientdatahtml-parsinghttp-requestspython-scrapingpython-web-scraperrequestsweb-crawlerweb-scrapingwebscrapingxpathdata-extraction
  Language:Python 311
• clauneck

  serpapi / clauneck

  A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.

  automationcommand-linecommand-line-tooldata-extractiondata-extractoremailemail-extract-with-proxyemail-extractionemail-extractoremail-marketingemail-scraperopen-sourcerubyrubygemserpsocial-media-scraperweb-crawlingwebscraping
  Language:Ruby 187

• molybdenum-99 / infoboxer

  Wikipedia information extraction library

  wikipediamediawikidata-extraction
  Language:Ruby 176
• sypht-python-client

  sypht-team / sypht-python-client

  A python client for the Sypht API

  data-extractioninformation-extractionapi-clientpythonpython3python3-librarysyphtsypht-python-clientsypht-apiinvoiceextractextract-fieldsextract-data-from-pdfreceipt-scannerpdf-parserreceipt-captureinvoice-parserreceipt-readerreceipt-scanningdocument-capture
  Language:Python 162

• dilawar / PlotDigitizer

  A Python utility to digitize plots.

  digitizationdata-extractionpython3image-processing
  Language:Python 155

• johnbumgarner / newspaper3_usage_overview

  This repository provides usage examples for the Python module Newspaper3k.

  beautifulsoupdata-extractionnewsnewspapernewspaper3knlp-parsingpythonpython-requestsscraping-websites
  Language:Python 148

• CambioML / any-parser

  Accurate, private and configurable document retrieval LLM

  data-extractiondocumentllmpdfprivacystructured-dataunstructured-data
  Language:Python 130

• nfx / go-htmltable

  Structured HTML table data extraction from URLs in Go that has almost no external dependencies

  godata-extractiongo-genericshtml
  Language:Go 122
• sayn

  173TECH / sayn

  Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

  analyticsetldata-modelingdata-engineeringdata-sciencepythonsqlautomationeltdata-extraction
  Language:Python 120

• villagecomputing / superpipe

  Superpipe - optimized LLM pipelines for structured data

  classificationdata-extractiondata-labelingllmllm-evaluationllm-optimizationstructured-data
  Language:Python 108

• hermit-crab / ScrapeMate

  Scraping assistant tool. Editing and maintaining CSS/XPath selectors across webpages.

  xpath-selectorscrapingcss-selectorextensionfirefox-extensionchrome-extensiondata-miningdata-extractiondevtools
  Language:JavaScript 105

• tech-engine / goscrapy

  GoScrapy: Harnessing Go's power for blazingly fast web scraping, inspired by Python's Scrapy framework.

  golanggoscraperscrapywebscraperspiderdata-extractionweb-crawlerwebscrappinggo-scrapy
  Language:Go 101

• Zubdata / Google-Maps-Scraper

  Google maps scraper with gui

  data-extractiongooglemapsgui-applicationleadsheetspythonweb-scraping-softwarescraperautomationbotgoogle-maps-scrapergoogle-maps-scraper-pythongooglemapsscraperweb-automationweb-botwebautomationwebbot
  Language:Python 100

• reincubate / ricloud

  Python client for Reincubate's ricloud API. Yes, it works with iOS 14 & iPhone 12 backups!

  icloud-apiicloud-accessiclouddata-extractioncloudkitpython-clientdata-recovery
  Language:Python 96

• sshniro / line-segmentation-algorithm-to-gcp-vision

  Line segmentation algorithm for Google Vision API.

  google-visionproposed-algorithmdata-extractioninvoicesegmentation
  Language:Kotlin 96

• chenkovsky / cyac

  High performance Trie and Ahocorasick automata (AC automata) Keyword Match & Replace Tool for python. Correct case insensitive implementation!

  automatacythondata-extractiondouble-array-triekeyword-extractionnlpsearchsearch-in-texttrie
  Language:Cython 95

• docwire / docwire

  DocWire SDK: Award-winning modern data processing in C++20. SourceForge Community Choice & Microsoft support. AI-driven processing. Supports nearly 100 data formats, including email boxes and OCR. Boost efficiency in text extraction, web data extraction, data mining, document analysis. Offline processing is possible for security and confidentiality

  apicclicpplinuxmacosparsingshellterminalwindowstensorflowsdktext-extractionmachine-learningartificial-intelligencedata-extractiontext-extraction-from-imagedata-processingtext-miningextract-transform-load
  Language:C++ 94

• dav009 / flash

  Golang Keyword extraction/replacement Datastructure using Tries instead of regexes

  textgolangsearchtriedata-extractiongotext-search
  Language:Go 89