language-identification

There are 9 repositories under language-identification topic.

googlesamples / mlkit
A collection of sample apps to demonstrate how to use Google's ML Kit APIs on Android and iOS
barcode barcode-scanner face-detection image-labeling object-detection text-recognition language-identification translation smart-reply mlkit mlkit-android google ml-kit mlkit-genai mlkit-genai-image-description mlkit-genai-proofreading mlkit-genai-rewriting mlkit-genai-summarization
Language:Java 3948
modelscope / 3D-Speaker
A Repository for Single- and Multi-modal Speaker Verification, Speaker Recognition and Speaker Diarization
campplus speaker-diarization speaker-verification voxceleb 3d-speaker eres2net language-identification modelscope cnceleb sdpn
Language:Python 2395
lingua-py
pemistahl / lingua-py
The most accurate natural language detection library for Python, suitable for short text and mixed-language text
nlp natural-language-processing language-detection language-recognition language-identification language-classification python-library
Language:Python 1493
lingua-go
pemistahl / lingua-go
The most accurate natural language detection library for Go, suitable for short text and mixed-language text
natural-language-processing language-detection language-recognition language-classification language-identification language-processing nlp nlp-machine-learning golang-library go language-modeling text-processing
Language:Go 1278
lingua-rs
pemistahl / lingua-rs
The most accurate natural language detection library for Rust, suitable for short text and mixed-language text
rust rust-library rust-crate language-detection language-classification language-recognition natural-language-processing nlp language-identification nlp-machine-learning language-processing
Language:Rust 948
lingua
pemistahl / lingua
The most accurate natural language detection library for Java and the JVM, suitable for long and short text alike
language-detection language-processing kotlin-library java-library nlp-library nlp nlp-machine-learning natural-language natural-language-processing android-library language-identification language-classification language-recognition
Language:Kotlin 770
echogarden-project / echogarden
Cross-platform speech toolset, used from the command-line or as a Node.js library. Includes a variety of engines for speech synthesis, speech recognition, forced alignment, speech translation, voice isolation, language detection and more.
forced-alignment language-identification speech speech-alignment speech-recognition speech-synthesis speech-to-text speech-translation text-to-speech language-detection source-separation command-line voice-isolation node-js
Language:TypeScript 349
apcode / tensorflow_fasttext
Simple embedding based text classifier inspired by fastText, implemented in tensorflow
fasttext tensorflow language-identification text-classifier
Language:Python 303
textpipe / textpipe
Textpipe: clean and extract metadata from text
nlp named-entities named-entity-recognition text-processing text-analysis language-identification
Language:Python 302
LlmKira / fast-langdetect
⚡️ 80x faster Fasttext language detection out of the box | Split text by language
fasttext i18n detect-languages languagedetector language-identification svc tts
Language:Python 234
vunb / vntk
Vietnamese NLP Toolkit for Node
vietnamese-nlp vietnamese-tokenizer natural-language-processing vietnamese vietnamese-text-classification language-identification named-entity-recognition tf-idf pos-tagging
Language:JavaScript 216
adbar / simplemma
Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
corpus-tools language-detection language-identification lemmatiser lemmatization lemmatizer low-resource-nlp morphological-analysis nlp tokenization tokenizer wordlist
Language:Python 154
cisnlp / GlotLID
💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023
language-detection language-detection-lib language-detection-library language-detector language-identification language-identification-toolkit language-identifier langid lid low-resource-languages low-resource-nlp multlingual language-classification language-recognition glot glotlid glotcc
Language:Python 126
HPI-DeepLearning / crnn-lid
Code for the paper Language Identification Using Deep Convolutional Recurrent Neural Networks
language-identification deep-learning computer-vision cnns keras
Language:Python 106
KrishnaDN / x-vector-pytorch
Implementation of the paper "Spoken Language Recognition using X-vectors" in Pytorch
language-identification language-recognition speech x-vector x-vector-pytorch
Language:Python 106
SpeechFlow-io / Spoken_language_identification
A TensorFlow-based spoken language identification
python speech tensorflow cnn language-identification language-recognition spoken-language-recognition
Language:Python 86
nitotm / efficient-language-detector-js
Fast and accurate natural language detection. Detector written in Javascript. Nito-ELD, ELD.
javascript language language-detection language-detector language-identification natural-language natural-language-processing nlp nodejs
Language:JavaScript 62
adbar / py3langid
Faster, modernized fork of the language identification tool langid.py
language-detection language-identification langid nlp whatlang detect-language language-recognition
Language:Python 55
microsoft / LID-tool
This code provides word level language identification tool for identifying language for individual words in Code-Mixed text. e.g. The text that includes words from two languages such as Hindi written in roman script, mixed with English.
language-identification natural-language-processing python3 mallet language-tags code-mixing code-switching linguistics
Language:Python 53
swshon / dialectID_e2e
End to End Dialect Identification using Convolutional Neural Network
dialect identification langugage-recognition i-vector dialectal speech mgb3 mgbchallenge multi-genre arabic arabic-language language-identification recognition dnn cnn lre sre
Language:Python 52
split-lang
DoodleBears / split-lang
✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and budoux
nlp python sentence-segmenter split detectlang i18n language-identification languagedetector tts langdetect pip pythonpackage fasttext budoux
Language:Jupyter Notebook 51
nitotm / efficient-language-detector
Fast and accurate natural language detection. Detector written in PHP. Nito-ELD, ELD.
language-detection language-detector natural-language natural-language-processing nlp php language language-identification language-classification
Language:PHP 49
py-lidbox / lidbox
End-to-end spoken language identification out of the box.
language-recognition language-identification tensorflow spoken-language-recognition spoken-language-identification audio-analysis speech deep-learning big-data
Language:Python 48
currentslab / fastlangid
fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)
cantonese-language traditional-chinese simplified-chinese identification fasttext language-identification language-identifier
Language:Python 39
rosette-api / python
Babel Street Analytics Client Library for Python
nlp machine-learning natural-language-processing text-analysis python entity-extraction name-generation language-detection language-identification categorization tokenization morphology lemmatization relation-extraction name-translation name-similarity sentiment-analysis text text-mining fuzzy-matching
Language:Python 38
zkmkarlsruhe / language-identification
Spoken Language Identification on Common Voice and AudioSet using Deep Learning
spoken-language-identification language-identification lid common-voice audioset intelligent-museum zkm
Language:Python 38
sagorbrur / codeswitch
CodeSwitch is a NLP tool, can use for language identification, pos tagging, name entity recognition, sentiment analysis of code mixed data.
codeswitch code-mixed nlp pos ner code-switching hindi-english spanish-english pos-tagging language-identification sentiment-analysis huggingface transformers
Language:Jupyter Notebook 35
mbanon / fastspell
Targetted language identifier, based on FastText and Hunspell.
fasttext hunspell language-identification nlp
Language:Python 34
UBC-NLP / afrolid
AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages.
african-languages deep-learning language-identification nlp toolkit dlnlp natural-langauge-processing ubc-nlp
Language:Python 31
hiredscorelabs / seqtolang
Multi-Langauge Identification
machine-learning deep-learning natural-language-processing language-identification
Language:Python 29
nipunmanral / Spoken-Language-Identification
Implement a GRU/LSTM model using Keras, and train it to classify the languages using MFCC features
gru lstm deep-learning neural-networks keras language-identification mfcc
Language:Python 25
dss-plugin-nlp-preparation
dataiku / dss-plugin-nlp-preparation
Dataiku DSS plugin to detect languages, correct misspellings, and clean text data 🧼
dataiku dss-plugin nlp natural-language-processing auto-correct spell-checker language-detection language-identification text-cleaning
Language:Python 22
floydhub / language-identification-template
Detect the languages from short pieces of text
floydhub language-identification language-detection keras tensorflow nlp neural-network deep-learning machine-learning
Language:Jupyter Notebook 22
igorsitdikov / lid_kaldi
language-recognition language-identification spoken-language-recognition spoken-language-identification kaldi kaldi-asr
Language:C++ 22
smola / language-dataset
Dataset for programming language identification.
language-detection dataset language-identification programming-language-identification
Language:Python 22
joshdevins / demo-es-lang-ident
Demo: Elasticsearch Language Identification
demo elasticsearch language-identification nlp search
Language:Python 19