sentence-tokenizer

There are 1 repository under sentence-tokenizer topic.

nipunsadvilkar / pySBD
🐍💯pySBD (Python Sentence Boundary Disambiguation) is a rule-based sentence boundary detection that works out-of-the-box.
python rule-based segmentation sentence sentence-boundary-detection sentence-tokenizer
Language:Python 732
neurosnap / sentences
A multilingual command line sentence tokenizer in Golang
sentence-tokenizer tokenizer cli sentences
Language:Go 424
vnlp
vngrs-ai / vnlp
State-of-the-art, lightweight NLP tools for Turkish language. Developed by VNGRS.
nlp turkish-nlp stemming morphological-analysis morphological-disambiguation named-entity-recognition dependency-parsing part-of-speech-tagging sentiment-analysis spelling-correction number-to-words deasciifier normalization stopword-removal sentence-tokenizer sentence-splitting word-embeddings word2vec fasttext deep-learning
Language:Python 239
megagonlabs / bunkai
Sentence boundary disambiguation tool for Japanese texts (日本語文境界判定器)
python sentence-tokenizer sentence-boundary-detection japanese
Language:Python 178
lfcipriani / punkt-segmenter
Ruby port of the NLTK Punkt sentence segmentation algorithm
nlp-library nltk punkt-segmenter ruby ruby-port rubynlp sentence-boundaries sentence-tokenizer tokenized-sentences
Language:Ruby 92
cbilgili / zemberek-nlp-server
Zemberek Türkçe NLP Java Kütüphanesi üzerine REST Docker Sunucu
nlp zemberek javascript rest spark part-of-speech-tagger sentence-tokenizer turkish docker turkish-language
Language:Java 74
wwwcojp / ja_sentence_segmenter
japanese sentence segmentation library for python
nlp sentence-boundary-detection rule-based python sentence-tokenizer
Language:Python 61
brandonrobertz / sentence-autosegmentation
Deep-learning based sentence auto-segmentation from unstructured text w/o punctuation
nlp deep-learning sentence-tokenizer
Language:Python 37
Flight-School / sentences
A command-line utility that splits natural language text into sentences.
cli macos nlp sentence-tokenizer swift
Language:Swift 37
ikegami-yukino / sengiri
Yet another sentence-level tokenizer for the Japanese text
japanese-language japanese-sentences sentence-tokenizer tokenizer
Language:Python 21
apdullahyayik / TrTokenizer
🧩 A simple sentence tokenizer.
sentence-tokenizer word-tokenizing word-segmentation turkish-nlp turkish-language regular-expression
Language:Python 20
lord-alfred / dnlp
📚 Сборник полезных штук из Natural Language Processing: Определение языка текста, Разделение текста на предложения, Получение основного содержимого из html документа
article-extracting article-extractor fasttext language-detection language-recognition nlp nlp-parsing nltk readability sentence-tokenizer text-processing
Language:Python 17
Foysal87 / bn_nlp
Bangla NLP toolkit.
bangla bangla-nlp bangla-word-embedding bangla-word2vec machine-learning nlp preprocessing sentence-embeddings sentence-similarity sentence-tokenizer sorting stemmer stopwords-removal tokenizer word-embeddings
Language:Python 11
bhattbhavesh91 / sentence-transformers-example
HuggingFace's Transformer models for sentence / text embedding generation.
huggingface huggingface-transformers huggingface-transformers-pipeline sentence-embeddings sentence-similarity sentence-tokenizer
Language:Jupyter Notebook 8
KMiNT21 / html2sent
HTML2SENT modifies HTML to improve sentences tokenizer quality
nlp nltk python sentence-segmentation sentence-tokenizer text-mining tokenizer
Language:Python 8
StarlangSoftware / Corpus
Corpus processing library
corpus-processing sentence-segmentation sentence-tokenizer turkish-sentence-segmentation turkish-sentence-tokenizer
Language:Java 6
yohamta / gosbd
A sentence boundary disambiguation library for Go. It is rule-based and works out-of-the-box.
natural-language-processing nlp-library sentence-boundary-detection sentence-tokenizer golang sentence-segmentation sentence-segmenter
Language:Go 6
mkartawijaya / hasami
A tool to perform sentence segmentation on Japanese text
japanese japanese-language nlp sentence-boundary-detection sentence-segmentation sentence-tokenizer
Language:Python 4
StarlangSoftware / Corpus-CPP
Corpus processing library
corpus-processing sentence-segmentation sentence-tokenizer turkish-sentence-segmentation turkish-sentence-tokenizer
Language:C++ 4
StarlangSoftware / Corpus-Py
Corpus processing library
corpus-processing sentence-segmentation sentence-tokenizer turkish-sentence-segmentation turkish-sentence-tokenizer
Language:Python 3
sichkar-valentyn / Machine_Learning_in_Python
Practical experiments on Machine Learning in Python. Processing of sentences and finding relevant ones, approximation of function with polynomials, function optimization
machine-learning sentence-tokenizer cosine-distance function-approximation polynomial-calculator function-optimization function-minimization
Language:Python 2
deepakrana47 / Sentence_tokenizer
Consist of Neural Network based sentence Tokenizer
sentence-tokenizer neural-network multilayer-perceptron-network
Language:Python 1
Musaddiq625 / Python-Projects
Some of my Python Projects
numpy matplotlib wordtoken-python sentence-tokenizer tokenizer python python3 pandas graph plot game guess random decision-tree-classification decision-tree-classifier sklearn prediction predict guess-the-number artificial-intelligence
Language:Python 1
rmjacobson / privacy-crawler-parser-tokenizer
Crawler, Parser, Sentence Tokenizer for online privacy policies. Intended to support ML efforts on policy language and verification.
privacy-policy html-parser sentence-tokenizer web-crawler web-crawler-python
Language:HTML 1
StarlangSoftware / Corpus-Js
Corpus Processing Library
sentence-tokenizer sentence-segmentation corpus-processing turkish-sentence-segmentation turkish-sentence-tokenizer
Language:TypeScript 1
victoryosiobe / kingchop
Kingchop ⚔️ is a JavaScript English based library for tokenizing text (chopping text). It uses vast rules for tokenizing, and you can adjust them easily.
javascript natural-language-processing nodejs paragraph-tokenizer sentence-tokenizer text-processing text-tokenization tokenizers word-tokenizer
Language:JavaScript 1
Aburraq / StanfordCoreNLP
My legal background gave me a deep appreciation for language's importance. It's not just words; it's a profound understanding woven into every case. This connection led me to coding, where I coded a potent pipeline system with Stanford CoreNLP.
java lemmatizer named-entity-recognition nlp oop partofspeech-tagger sentence-tokenizer sentiment-analysis stanfordnlp tokenizer
Language:Java 0
Arboghast / Gutenberg-parser
A homemade sentence tokenizer designed for Project Gutenberg books
nlp-parsing sentence-tokenizer gutenberg
Language:R 0
nature-of-eu-rules / data-preprocessing
Document preprocessing scripts for the Nature of EU Rules project
european-union html law legislation lexnlp pdf preprocessing pymupdf python sentence-segmentation sentence-tokenizer text tokenization
Language:Python 0
paaila / language
Language processing for better query answering
machine-learning natural-language-processing sentence-segmentation sentence-tokenizer sklearn-classify
Language:Python 0
elifftosunn / textDataClean
Kirli veri çekildiğinde ön işleme adımlarına gerek kalmadan model eğitimi için hazır hale getirmek amacıyla yapılan uygulamadır.
corpus deasciifier morphological-analysis ngram nltk numpy pandas sentence-embedding sentence-tokenizer stemmer stopwords string turkish turkish-sentence-tokenizer word-tokenizer
quocthang0507 / VietnameseNaturalLanguageProcessing
Vietnamese Natural Language Processing
nlp natural-language-processing sentence-tokenizer word-tokenizer
Language:Jupyter Notebook
StarlangSoftware / Corpus-CS
Corpus processing library
sentence-tokenizer corpus-processing sentence-segmentation turkish-sentence-segmentation turkish-sentence-tokenizer
Language:C#
StarlangSoftware / Corpus-Cy
Corpus Processing Library
sentence-tokenizer sentence-segmentation corpus-processing turkish-sentence-tokenizer turkish-sentence-segmentation
Language:Cython
StarlangSoftware / Corpus-Swift
Corpus processing library
sentence-tokenizer corpus-processing sentence-segmentation turkish-sentence-segmentation turkish-sentence-tokenizer
Language:Swift
zainmujahid / Longest-Common-Subsequence
This repository contains python script for calculating Longest Common Subsequences (LSC) between tokenized URDU sentences.
longest-common-subsequence sentence-similarity sentence-tokenizer urdu-nlp lsc python
Language:Jupyter Notebook

sentence-tokenizer

nipunsadvilkar / pySBD

neurosnap / sentences

vngrs-ai / vnlp

megagonlabs / bunkai

lfcipriani / punkt-segmenter

cbilgili / zemberek-nlp-server

wwwcojp / ja_sentence_segmenter

brandonrobertz / sentence-autosegmentation

Flight-School / sentences

ikegami-yukino / sengiri

apdullahyayik / TrTokenizer

lord-alfred / dnlp

Foysal87 / bn_nlp

bhattbhavesh91 / sentence-transformers-example

KMiNT21 / html2sent

StarlangSoftware / Corpus

yohamta / gosbd

mkartawijaya / hasami

StarlangSoftware / Corpus-CPP

StarlangSoftware / Corpus-Py

sichkar-valentyn / Machine_Learning_in_Python

deepakrana47 / Sentence_tokenizer

Musaddiq625 / Python-Projects

rmjacobson / privacy-crawler-parser-tokenizer

StarlangSoftware / Corpus-Js

victoryosiobe / kingchop

Aburraq / StanfordCoreNLP

Arboghast / Gutenberg-parser

nature-of-eu-rules / data-preprocessing

paaila / language

elifftosunn / textDataClean

quocthang0507 / VietnameseNaturalLanguageProcessing

StarlangSoftware / Corpus-CS

StarlangSoftware / Corpus-Cy

StarlangSoftware / Corpus-Swift

zainmujahid / Longest-Common-Subsequence