corpus-builder

There are 3 repositories under corpus-builder topic.

• adbar / trafilatura

  Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

  web-scrapingtext-extractionnlphtml2texttext-miningcrawlertext-cleaningtext-preprocessingarticle-extractorreadabilityscrapingnews-crawlerteihtml-to-markdowncorpus-buildercorpus-toolsrss-feednews-aggregatorragllm
  Language:Python 4671

• google / corpuscrawler

  Crawler for linguistic corpora

  corpus-buildercorpus-linguisticscrawlinglinguisticsminority-language
  Language:Python 208

• praaline / Praaline

  Praaline is an open-source system to manage, annotate, visualise and analyse spoken language corpora

  visualisationannotationslinguisticscorpuscorpus-linguisticscorpus-toolscorpus-builderspeech-processingspeech-analysisspoken-language-processing
  Language:C 30

• carlfm01 / librivox-tools

  Collector and speech cutter for librivox audiobooks

  speech-to-textcorpus-toolscorpus-builderlibrivoxdata-collector
  Language:C# 22

• uma-pi1 / OPIEC-pipeline

  open-information-extractiontext-processingcorpus-datacorpus-toolscorpus-linguisticscorpus-processingcorpus-buildercorpus-generatorwikipediawikiinformation-extractionbig-databigdatanatural-language-processingnatural-language-understandingnlpnlp-resourcesnlp-datasetsnlp-apiswikipedia-corpus
  Language:Java 14

• dohliam / ebook-corpus

  Ebook Corpus - A parser and extractor for electronic books

  corpuscorpus-linguisticscorpus-builderebooksepubmobifb2ebook-parsing
  Language:Ruby 7

• AndyTheFactory / article-extraction-dataset

  Article title, authors, date and body extraction dataset.

  article-extractorcorpuscorpus-buildercorpus-toolsdatasetdatasetshtml-to-markdownhtml2textnewsnews-aggregatornews-crawlerreadabilityscrapingscraping-websitestext-cleaningtext-extractiontext-miningtext-preprocessingweb-scraping
  Language:HTML 6

• thecsw / katya-dev

  Katya or The Liberated Corpus a text corpus that allows you to request and scrape any web resource!

  corpuscorpus-analysiscorpus-buildercorpus-generatorcorpus-linguisticscorpus-processingrussianrussian-literaturetaggertext-corpus
  Language:Go 6

• FerreroJeremy / Plagiarized-Corpus-Generator

  A corpus builder for evaluation of plagiarism detection tools

  corpus-generatorcorpus-builderplagiarism
  Language:PHP 2

• jhlopesalves / CorpusAid

  Automated text preprocessing pipeline for large corpora. Features customizable filters for diacritics, stop words, punctuation, and regex.

  corpus-buildercorpus-linguisticscorpus-processingcorpus-toolsdata-cleaningdata-cleaning-automationnatural-language-processingpythonregextext-preprocessing
  Language:Python 1

• tubone24 / askfm-qa-crawler

  Crawl Ask.fm QA lists and create corpus for ML.

  askfmseleniumchromedrivercrawlercorpus-builder
  Language:Python 1

• writecrow / crow_backend

  The canonical resources to build the backend for a corpus/repository management framework for Crow, the Corpus and Repository of Writing

  apibackendcorpuscorpus-buildercorpus-generatorcorpus-linguisticsnatural-language-processing
  Language:PHP 1
• crow_frontend

  writecrow / crow_frontend

  The user interface for the Corpus & Repository of Writing, built in Angular

  angularcorporacorpuscorpus-buildercorpus-linguisticsnatural-language-processing
  Language:TypeScript 1

• adpaczek / chatbot

  Chatbot in Polish language, trained on movie subtitles collected using web scraping, based on Transformer architecture.

  chatbotcorpus-buildernlppolish-nlutransformerweb-scraping
  Language:Jupyter Notebook 0

• CristinaGHolgado / vikitext

  Extract text from Vikidia/Wikipedia articles [fr]

  corpus-buildercorpusreadabilitytext-simplificationfrench-nlpvikidiawikipedia-scraper
  Language:Python 0

• IDS-Mannheim / Wikipedia-Corpus-Builder

  Builds Wikipedia corpora in I5 (a TEI-based format)

  wikipediacorpus-builderwikipedia-corpusxmltei
  Language:Java 0

• sorinmarti / fruechtekorb

  This is a text corpus management system for the german linguistic department of the university of Basel.

  linguisticscorpus-linguisticscorpuscorpus-builder
  Language:PHP 0

• binayachaudari / Corpus-Development-Software

  Corpus Development Software for Machine Translation

  machine-translationcorpus-buildermachine-learning
  Language:JavaScript

• c0ntradicti0n / CorpusCookApp

  App and Scripts working with the corpus-builder CorpusCook, to have a corpus updated with corrected wrong predictions

  python3kivy-applicationtwistedampnlp-machine-learningcorpus-linguisticscorpus-builder
  Language:Python