text-processing

There are 24 repositories under text-processing topic.

learnbyexample / Command-line-text-processing
:zap: From finding text to search and replace, from sorting to beautifying text and more :art:
awk command-line ebook grep linux perl regex ruby sed text-processing
Language:Shell 10175
google / diff-match-patch
Diff Match Patch is a high-performance library in multiple languages that manipulates plain text.
diff difference match patch text-processing
Language:Python 7410
chmln / sd
Intuitive find & replace CLI (sed alternative)
cli command-line regex rust terminal text-processing
Language:Rust 5777
PyMuPDF
pymupdf / PyMuPDF
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
data-science epub extract-data font mupdf ocr pdf pdf-documents pymupdf python table-extraction tesseract text-processing text-shaping xps
Language:Python 5102
fastnlp / fastNLP
fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
chinese-nlp deep-learning natural-language-processing nlp-library nlp-parsing text-classification text-processing
Language:Python 3061
pyparsing / pyparsing
Python library for creating PEG parsers
parser-combinators parsing parsing-expression-grammar parsing-library peg-parsers python python-2 python-3 python2 python3 text-processing
Language:Python 2191
kk7nc / Text_Classification
Text Classification Algorithms: A Survey
boosting-algorithms conditional-random-fields convolutional-neural-networks decision-trees deep-belief-network deep-learning deep-neural-network dimensionality-reduction document-classification hierarchical-attention-networks k-nearest-neighbours logistic-regression naive-bayes-classifier nlp-machine-learning random-forest recurrent-neural-networks rocchio-algorithm support-vector-machines text-classification text-processing
Language:Python 1795
birchb1024 / frangipanni
Program to convert lines of text into a tree structure.
go golang text-processing tree-structure
Language:Go 1199
roshan-research / hazm
Persian NLP Toolkit
dependency-parser embeddings farsi lemmatization natural-language-processing nlp normalization persian persian-nlp pos-tagging python text-processing tokenizer
Language:Python 1188
lingua-go
pemistahl / lingua-go
The most accurate natural language detection library for Go, suitable for short text and mixed-language text
natural-language-processing language-detection language-recognition language-classification language-identification language-processing nlp nlp-machine-learning golang-library go language-modeling text-processing
Language:Go 1157
BurntSushi / aho-corasick
A fast implementation of Aho-Corasick in Rust.
aho-corasick substring-matching finite-state-machine text-processing search
Language:Rust 1006
helix-editor / nucleo
A fast and convenient fuzzy matcher library for rust
fuzzy-matching fuzzy-search performance rust text-processing
Language:Rust 835
sstadick / hck
A sharp cut(1) clone.
command-line rust text-processing
Language:Rust 693
cbaziotis / ekphrasis
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
nlp nlp-library semeval spell-corrector spelling-correction text-processing text-segmentation tokenization tokenizer word-normalization word-segmentation
Language:Python 660
derek73 / python-nameparser
A simple Python module for parsing human names into their individual components
python python-module text-parser text-processing
Language:Python 651
abadojack / whatlanggo
Natural language detection library for Go
language go nlp text-processing
Language:Go 638
open-korean-text / open-korean-text
Open Korean Text Processor - An Open-source Korean Text Processor
korean korean-text-processing natural-language-processing text-processing tokenizer korean-tokenizer
Language:Scala 608
ChenghaoMou / text-dedup
All-in-one text de-duplication
data-processing de-duplication nlp text-processing
Language:Python 587
proycon / pynlpl
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).
nlp python computational-linguistics linguistics library folia machine-learning language-modelling search-algorithms evaluation-metrics text-processing nlp-library natural-language-processing
Language:Python 479
wenet-e2e / WeTextProcessing
Text Normalization & Inverse Text Normalization
normalization production-ready text-processing
Language:Python 445
pyarabic
linuxscout / pyarabic
pyarabic
arabic-language nlp-library text-processing
Language:Python 438
andrewbihl / bsed
Simple SQL-like syntax on top of Perl text processing.
sed perl grep awk python text-processing csv domain-specific-language
Language:Python 409
PyKoSpacing
haven-jeon / PyKoSpacing
Automatic Korean word spacing with Python
korean-nlp nlp text-processing spacing
Language:Python 395
airbnb / artificial-adversary
🗣️ Tool to generate adversarial text examples and test machine learning models against them
machine-learning classification python python3 python2 text text-mining adversarial-examples spam spam-filtering spam-detection spam-classification text-classification text-analysis data-science data-mining text-processing black-box-benchmarking black-box-attacks metrics
Language:Python 393
BurntSushi / regex-automata
A low level regular expression library that uses deterministic finite automata.
automata automaton dfa nfa regex regex-engine regexp rust text-processing
Language:Rust 351
ikegami-yukino / jaconv
Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku
japanese-language preprocessing text-processing japanese-kana pure-python character-converter julius transliteration
Language:Python 305
gagolews / stringi
Fast and portable character string processing in R (with the Unicode ICU)
icu icu4c natural-language-processing nlp r regex regexp string-manipulation stringi stringr text text-processing tidy-data unicode
Language:C++ 300
textpipe / textpipe
Textpipe: clean and extract metadata from text
nlp named-entities named-entity-recognition text-processing text-analysis language-identification
Language:Python 300
lukaszliniewicz / Pandrator
Pandrator aspires to be a user-friendly app with a graphical interface and a one-click installer that creates high-quality speech from text in multiple languages (audiobooks, speech synchronised with subtitles and more) using local models (XTTS, Silero or VoiceCraft), plus voice cloning, LLM pre-processing, RVC enhancement, and automatic evaluation
audiobook audiobook-creator audiobook-maker audiobooks text-processing text-to-speech customtkinterprojects llm nisqa rvc tkinter-gui xtts xttsv2 silero voice-cloning voicecraft dubbing pdf-to-audio subtitle-to-speech subtitle-to-voice
Language:Python 279
RandyPen / TextCluster
短文本聚类预处理模块 Short text cluster
cluster nlp text-cluster text-clustering text-mining text-processing
Language:Python 266
rust-unic
open-i18n / rust-unic
UNIC: Unicode and Internationalization Crates for Rust
unicode internationalization text-processing crates rust cldr locale-data unic unicode-characters unicode-algorithms
Language:Rust 234
himkt / konoha
🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.
nlp text-processing mecab kytea sudachi sentencepiece natural-language-processing japanese janome
Language:Python 226
catatsuy / purl
Streamlining Text Processing
grep-like regexp sed text-processing
Language:Go 212
larrykollar / Unix-Text-Processing
Recreated sources for the book "UNIX Text Processing," published in 1987.
gnu-troff unix utp utp-revival groff text-processing publishing formatting
Language:Roff 209
bytesparadise / libasciidoc
A Golang library for processing Asciidoc files.
asciidoc golang library text-processing
Language:Go 200
daac-tools / daachorse
🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure in Rust.
search finite-state-machine text-processing aho-corasick substring-matching rust double-array no-std
Language:Rust 199

text-processing

learnbyexample / Command-line-text-processing

google / diff-match-patch

chmln / sd

pymupdf / PyMuPDF

fastnlp / fastNLP

pyparsing / pyparsing

kk7nc / Text_Classification

birchb1024 / frangipanni

roshan-research / hazm

pemistahl / lingua-go

BurntSushi / aho-corasick

helix-editor / nucleo

sstadick / hck

cbaziotis / ekphrasis

derek73 / python-nameparser

abadojack / whatlanggo

open-korean-text / open-korean-text

ChenghaoMou / text-dedup

proycon / pynlpl

wenet-e2e / WeTextProcessing

linuxscout / pyarabic

andrewbihl / bsed

haven-jeon / PyKoSpacing

airbnb / artificial-adversary

BurntSushi / regex-automata

ikegami-yukino / jaconv

gagolews / stringi

textpipe / textpipe

lukaszliniewicz / Pandrator

RandyPen / TextCluster

open-i18n / rust-unic

himkt / konoha

catatsuy / purl

larrykollar / Unix-Text-Processing

bytesparadise / libasciidoc

daac-tools / daachorse