text-processing

There are 24 repositories under text-processing topic.

learnbyexample / Command-line-text-processing
:zap: From finding text to search and replace, from sorting to beautifying text and more :art:
awk command-line ebook grep linux perl regex ruby sed text-processing
Language:Shell 10177
google / diff-match-patch
Diff Match Patch is a high-performance library in multiple languages that manipulates plain text.
difference diff match patch text-processing
Language:Python 7410
chmln / sd
Intuitive find & replace CLI (sed alternative)
cli command-line regex rust terminal text-processing
Language:Rust 5777
PyMuPDF
pymupdf / PyMuPDF
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
data-science epub extract-data font mupdf ocr pdf pdf-documents pymupdf python table-extraction tesseract text-processing text-shaping xps
Language:Python 5096
fastnlp / fastNLP
fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
natural-language-processing deep-learning nlp-library nlp-parsing chinese-nlp text-classification text-processing
Language:Python 3061
pyparsing / pyparsing
Python library for creating PEG parsers
python python2 python3 python-2 python-3 parser-combinators parsing-expression-grammar parsing parsing-library text-processing peg-parsers
Language:Python 2189
kk7nc / Text_Classification
Text Classification Algorithms: A Survey
boosting-algorithms conditional-random-fields convolutional-neural-networks decision-trees deep-belief-network deep-learning deep-neural-network dimensionality-reduction document-classification hierarchical-attention-networks k-nearest-neighbours logistic-regression naive-bayes-classifier nlp-machine-learning random-forest recurrent-neural-networks rocchio-algorithm support-vector-machines text-classification text-processing
Language:Python 1795
birchb1024 / frangipanni
Program to convert lines of text into a tree structure.
go golang text-processing tree-structure
Language:Go 1199
roshan-research / hazm
Persian NLP Toolkit
natural-language-processing python persian nlp persian-nlp dependency-parser embeddings text-processing tokenizer lemmatization farsi normalization pos-tagging
Language:Python 1187
lingua-go
pemistahl / lingua-go
The most accurate natural language detection library for Go, suitable for short text and mixed-language text
go golang-library language-classification language-detection language-identification language-modeling language-processing language-recognition natural-language-processing nlp nlp-machine-learning text-processing
Language:Go 1157
BurntSushi / aho-corasick
A fast implementation of Aho-Corasick in Rust.
aho-corasick finite-state-machine search substring-matching text-processing
Language:Rust 1006
helix-editor / nucleo
A fast and convenient fuzzy matcher library for rust
fuzzy-matching fuzzy-search performance rust text-processing
Language:Rust 831
sstadick / hck
A sharp cut(1) clone.
command-line rust text-processing
Language:Rust 693
cbaziotis / ekphrasis
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
nlp text-processing nlp-library spelling-correction tokenizer tokenization word-segmentation word-normalization spell-corrector text-segmentation semeval
Language:Python 660
derek73 / python-nameparser
A simple Python module for parsing human names into their individual components
python python-module text-parser text-processing
Language:Python 651
abadojack / whatlanggo
Natural language detection library for Go
language go nlp text-processing
Language:Go 638
open-korean-text / open-korean-text
Open Korean Text Processor - An Open-source Korean Text Processor
korean korean-text-processing natural-language-processing text-processing tokenizer korean-tokenizer
Language:Scala 608
ChenghaoMou / text-dedup
All-in-one text de-duplication
data-processing de-duplication nlp text-processing
Language:Python 586
proycon / pynlpl
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).
nlp python computational-linguistics linguistics library folia machine-learning language-modelling search-algorithms evaluation-metrics text-processing nlp-library natural-language-processing
Language:Python 479
wenet-e2e / WeTextProcessing
Text Normalization & Inverse Text Normalization
normalization production-ready text-processing
Language:Python 445
pyarabic
linuxscout / pyarabic
pyarabic
arabic-language nlp-library text-processing
Language:Python 438
andrewbihl / bsed
Simple SQL-like syntax on top of Perl text processing.
sed perl grep awk python text-processing csv domain-specific-language
Language:Python 409
PyKoSpacing
haven-jeon / PyKoSpacing
Automatic Korean word spacing with Python
korean-nlp nlp text-processing spacing
Language:Python 395
airbnb / artificial-adversary
🗣️ Tool to generate adversarial text examples and test machine learning models against them
machine-learning classification python python3 python2 text text-mining adversarial-examples spam spam-filtering spam-detection spam-classification text-classification text-analysis data-science data-mining text-processing black-box-benchmarking black-box-attacks metrics
Language:Python 393
BurntSushi / regex-automata
A low level regular expression library that uses deterministic finite automata.
automata automaton dfa nfa regex regex-engine regexp rust text-processing
Language:Rust 351
ikegami-yukino / jaconv
Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku
character-converter japanese-kana japanese-language julius preprocessing pure-python text-processing transliteration
Language:Python 304
gagolews / stringi
Fast and portable character string processing in R (with the Unicode ICU)
icu icu4c natural-language-processing nlp r regex regexp string-manipulation stringi stringr text text-processing tidy-data unicode
Language:C++ 300
textpipe / textpipe
Textpipe: clean and extract metadata from text
language-identification named-entities named-entity-recognition nlp text-analysis text-processing
Language:Python 300
lukaszliniewicz / Pandrator
Pandrator aspires to be a user-friendly app with a graphical interface and a one-click installer that creates high-quality speech from text in multiple languages (audiobooks, speech synchronised with subtitles and more) using local models (XTTS, Silero or VoiceCraft), plus voice cloning, LLM pre-processing, RVC enhancement, and automatic evaluation
audiobook audiobook-creator audiobook-maker audiobooks text-processing text-to-speech customtkinterprojects llm nisqa rvc tkinter-gui xtts xttsv2 silero voice-cloning voicecraft dubbing pdf-to-audio subtitle-to-speech subtitle-to-voice
Language:Python 277
RandyPen / TextCluster
短文本聚类预处理模块 Short text cluster
cluster nlp text-cluster text-clustering text-mining text-processing
Language:Python 266
rust-unic
open-i18n / rust-unic
UNIC: Unicode and Internationalization Crates for Rust
unicode internationalization text-processing crates rust cldr locale-data unic unicode-characters unicode-algorithms
Language:Rust 234
himkt / konoha
🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.
nlp text-processing mecab kytea sudachi sentencepiece natural-language-processing japanese janome
Language:Python 226
catatsuy / purl
Streamlining Text Processing
grep-like regexp sed text-processing
Language:Go 212
larrykollar / Unix-Text-Processing
Recreated sources for the book "UNIX Text Processing," published in 1987.
gnu-troff unix utp utp-revival groff text-processing publishing formatting
Language:Roff 209
bytesparadise / libasciidoc
A Golang library for processing Asciidoc files.
golang library asciidoc text-processing
Language:Go 199
daac-tools / daachorse
🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure in Rust.
aho-corasick double-array finite-state-machine no-std rust search substring-matching text-processing
Language:Rust 199

text-processing

learnbyexample / Command-line-text-processing

google / diff-match-patch

chmln / sd

pymupdf / PyMuPDF

fastnlp / fastNLP

pyparsing / pyparsing

kk7nc / Text_Classification

birchb1024 / frangipanni

roshan-research / hazm

pemistahl / lingua-go

BurntSushi / aho-corasick

helix-editor / nucleo

sstadick / hck

cbaziotis / ekphrasis

derek73 / python-nameparser

abadojack / whatlanggo

open-korean-text / open-korean-text

ChenghaoMou / text-dedup

proycon / pynlpl

wenet-e2e / WeTextProcessing

linuxscout / pyarabic

andrewbihl / bsed

haven-jeon / PyKoSpacing

airbnb / artificial-adversary

BurntSushi / regex-automata

ikegami-yukino / jaconv

gagolews / stringi

textpipe / textpipe

lukaszliniewicz / Pandrator

RandyPen / TextCluster

open-i18n / rust-unic

himkt / konoha

catatsuy / purl

larrykollar / Unix-Text-Processing

bytesparadise / libasciidoc

daac-tools / daachorse