low-resource-languages

There are 6 repositories under low-resource-languages topic.

RichardLitt / low-resource-languages
Resources for conservation, development, and documentation of low resource (human) languages.
endangered-languages natural-language language-resources human-language natural-language-processing language-learning language-documentation resourced-languages awesome-list awesome list minority-language low-resource-languages lrls nlp
Language:TeX 372
csebuetnlp / xl-sum
This repository contains the code, data, and models of the paper titled "XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages" published in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.
abstractive-text-summarization text-summarization text-summarisation abstractive-summarization multilingual multilinguality multilingual-text-summarization low-resource-languages dataset summarization-dataset summarization-corpora text-summarization-dataset text-summarization-model low-resource-summarization low-resource-text-summarizarion multilingual-summarization machine-learning deep-learning
Language:Python 243
csebuetnlp / banglanmt
This repository contains the code and data of the paper titled "Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation" published in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), November 16 - November 20, 2020.
bangla-nlp machine-translation parallel-corpus parallel-corpora neural-machine-translation bangla-dataset-machine-translation bangla-machine-translation low-resource-languages emnlp-2020 low-resource-nlp low-resource-machine-translation
Language:Python 144
Andrews2017 / africanlp-public-datasets
A repository for publicly/freely available Natural Language Processing (NLP) datasets for African languages.
african-languages low-resource-languages natural-language-processing datasets
81
cisnlp / GlotLID
GlotLID v3: Language Identification with Support for More Than 2000 Labels (EMNLP 2023).
language-detection language-detection-lib language-detection-library language-detector language-identification language-identification-toolkit language-identifier langid lid low-resource-languages low-resource-nlp multlingual language-classification language-recognition glot
Language:Python 67
jcblaisecruz02 / Filipino-Text-Benchmarks
Open-source benchmark datasets and pretrained transformer models in the Filipino language.
transformer transfer-learning bert tagalog filipino corpus benchmark deep-learning tagalog-transformers text-classification electra electra-models nli low-resource-languages
Language:Python 55
kbatsuren / CogNet
CogNet: a large-scale, high-quality cognate database for 338 languages, 1.07M words, and 8.1 million cognates
cognate multilinguality cross-lingual-simialrity wordnet language-resources low-resource-languages corpus-linguistics bilingual-lexicon-extraction bilingual-lexicon-induction cross-lingual-transfer
40
alexandra-chron / relm_unmt
Python source code for EMNLP 2020 paper "Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT".
transfer-learning language-models cross-lingual pretraining low-resource-languages unsupervised-machine-translation residual-adapters
Language:Python 35
Rumeysakeskin / Turkish-Text-to-Speech
Speech synthesis (TTS) in low-resource languages by training from scratch with Fastpitch and fine-tuning with HifiGan
fastpitch hifigan low-resource-languages pytorch speech-synthesis tts phonetical-conversion turkish-text-to-speech nvidia-docker spectrogram-generator nvidia-nemo waveform-generator
Language:Python 35
ljvmiranda921 / calamanCy
NLP pipelines for Tagalog using spaCy
computational-linguistics low-resource-languages low-resource-nlp machine-learning natural-language-processing ner nlp spacy
Language:Python 34
cdli-gh / Semi-Supervised-NMT-for-Sumerian-English
Exploring the Limits of Low-Resource Neural Machine Translation
backtranslation low-resource-languages nmt semi-supervised transformers translation unsupervised xlm
Language:Jupyter Notebook 31
csikasote / BembaSpeech
This is an ASR corpus for Bemba language. It contains read speech from diverse publicly available Bemba sources; Literature Books, Radio/TV shows transcripts, Youtube Video transcripts, Online sources. The corpus has 14, 438 utterances culminating into over 24 hours of speech.
automatic-speech-recognition bemba low-resource-languages
28
Kartikaggarwal98 / Indian_ParallelCorpus
Curated list of publicly available parallel corpus for Indian Languages
machinetranslation indian-languages nlp corpus parallel-corpus parallel-corpora multilingual-translation low-resource-machine-translation low-resource-languages neural-machine-translation
28
Rumeysakeskin / Turkish-Speech-to-Text
Fine-tuning for automatic speech recognition on low-resource languages with character-based CTC model
speech-to-text quartz-net stt-inference onnxruntime automatic-speech-recognition low-resource-languages wer fine-tuning speech-data-augmentation
Language:Jupyter Notebook 28
hausanlp / NaijaSenti
This is a repository for NaijaSenti. A Lacuna Funded Project for the development of sentiment corpus for four Nigerian languages: Igbo, Hausa, Yoruba and Pidgin.
african-languages dataset hausa hausa-nlp hausanlp igbo igbo-language low-resource-languages low-resource-nlp nigeria nigerian-data nlp sentiment sentiment-analysis sentiment-classification sentiment-data yoruba yorubaname-dictionary
25
charlesliucn / LanMIT
📖 LanMIT: A Toolkit for Improving Language Models in Low-resourced Speech Recognition based on Kaldi.
low-resource-languages language-modeling speech-recognition speech-to-text keyword-spotting kaldi-asr
Language:C++ 21
RichardLitt / thesis
My thesis on "Open Source Code and Low Resource Languages" for an MSc in Language Science and Technology at Saarland University
thesis dissertation endangered-languages saarland saarland-university lrl low-resource-languages nlp nlproc
Language:TeX 20
vad-sli-asr
CoEDL / vad-sli-asr
A pipeline to isolate and transcribe one language in mixed-language speech
automatic-speech-recognition endangered-languages low-resource-languages spoken-language-identification voice-activity-detection
Language:Python 19
Aditi138 / EntityTargetedActiveLearning
nlp low-resource-languages named-entity-recognition transfer-learning active-learning
Language:Python 17
alecokas / BiLatticeRNN-Confidence
Confidence Estimation for Black Box Automatic Speech Recognition Systems Using Lattice Recurrent Neural Networks https://arxiv.org/abs/1910.11933 or https://ieeexplore.ieee.org/document/9053264
attention lstm speech-processing confidence-scores confidence-estimation pytorch confusion-networks low-resource-languages latticernn lattice lattices pytorch-implementation confidence-estimates speech-recognition asr
Language:Python 15
khuangaf / CONCRETE
Official implementation of "CONCRETE: Improving Cross-lingual Fact Checking with Cross-lingual Retrieval" (COLING'22)
cross-lingual-transfer fact-checking retrieval low-resource-languages multilinguality
Language:Python 14
surafelml / Afro-NMT
LOW-RESOURCE NEURAL MACHINE TRANSLATION: A BENCHMARK FOR FIVE AFRICAN LANGUAGES
low-resource-languages multilingaul neural-machine-translation transfer-learning
Language:Shell 14
jcblaisecruz02 / Tagalog-fake-news
Fake news detection in Filipino via Multitask Transfer Learning
bert bert-model tagalog filipino low-resource-languages transformer nlp deep-learning multitask-learning
13
unza-speech-lab / zambezi-voice
Repository for multilingual speech data resources for native languages of Zambia.
low-resource-languages speech-recognition speech-to-text zambia
13
dmatekenya / Chichewa-Speech2Text
Automated Speech Recognition for Chichewa.
asr chichewa low-resource-languages nlp
Language:Jupyter Notebook 12
jhdeov / interlingual-MFA
Workflow for forced alignment between languages
forced-alignment low-resource-languages montreal-forced-aligner multilingual-alignment cross-language cross-language-alignment
Language:Python 12
luciusssss / mc2_corpus
MC^2: A Multilingual Corpus of Minority Languages in China (Tibetan, Uyghur, Kazakh, and Mongolian)
corpus kazakh low-resource-languages low-resource-nlp mongolian multilingual natural-language-processing tibetan uyghur tibetan-nlp
Language:Python 12
IgnatiusEzeani / IGBONLP
This is a repository for the IGBONLP Project.
igbo-language nlp machine-translation deep-learning low-resource-languages
Language:Modula-3 11
Rumeysakeskin / ASR-fine-tuning-for-low-resource-languages
Transfer learning for ASR with subword encoding CTC model (NVIDIA NeMo Citrinet) on low-resource languages
asr ctc-model fine-tuning google-sentencepiece low-resource-languages nvidia-nemo tokenizer transfer-learning citrinet
Language:Jupyter Notebook 11
clefourrier / CopperMT
[ACL 2021, Findings] Cognate Prediction Per Machine Translation
acl2021 cognate-prediction cognates nmt smt machine-translation low-resource-languages low-resource-machine-translation fairseq
Language:JavaScript 10
harmanpreet93 / low-resource-machine-translation
Low resource machine translation using Transformers and Iterative Back translation
back-translation bert-embeddings low-resource-languages machine-translation nlp nmt transformer-models
Language:Python 10
fajri91 / minangNLP
Minangkabau NLP corpus. PACLIC 2020
sentiment-analysis machine-translation corpus minangkabau-language nlp bert low-resource-languages indonesian-language ethnicity paclic
Language:Python 9
ruoyuxie / noisy_parallel_data_alignment
Enhanced awesome-align for low-resource languages and noise simulation: https://arxiv.org/abs/2301.09685
low-resource-languages low-resource-nlp noise noisy-data nueral-machine-translation ocr ocr-text word-aligner word-alignment
Language:Python 9
tafseer-nayeem / BengaliReadability
[AAAI 2021] - Simple or Complex? Learning to Predict Readability of Bengali Texts.
bengali-nlp bengali-natural-language-processing bengali-language-processing bengali-readability bengali-readability-analysis bengali-readability-prediction readability-dataset bengali-readability-dataset low-resource-languages
Language:Python 9
alecokas / swahili-text-gcn
Graph Convolutional Network for Swahili News Classification: https://arxiv.org/abs/2103.09325
gcn python pytorch swahili graph-neural-networks natural-language-processing semi-supervised-learning graph-convolutional-networks text-classification low-resource-languages
Language:Jupyter Notebook 8
ofdn / OpenSpeaks-Before-AI
A set of frameworks for creating the AI/ML building blocks for low-resource languages.
languages low-resource-languages ai ml
8

low-resource-languages

RichardLitt / low-resource-languages

csebuetnlp / xl-sum

csebuetnlp / banglanmt

Andrews2017 / africanlp-public-datasets

cisnlp / GlotLID

jcblaisecruz02 / Filipino-Text-Benchmarks

kbatsuren / CogNet

alexandra-chron / relm_unmt

Rumeysakeskin / Turkish-Text-to-Speech

ljvmiranda921 / calamanCy

cdli-gh / Semi-Supervised-NMT-for-Sumerian-English

csikasote / BembaSpeech

Kartikaggarwal98 / Indian_ParallelCorpus

Rumeysakeskin / Turkish-Speech-to-Text

hausanlp / NaijaSenti

charlesliucn / LanMIT

RichardLitt / thesis

CoEDL / vad-sli-asr

Aditi138 / EntityTargetedActiveLearning

alecokas / BiLatticeRNN-Confidence

khuangaf / CONCRETE

surafelml / Afro-NMT

jcblaisecruz02 / Tagalog-fake-news

unza-speech-lab / zambezi-voice

dmatekenya / Chichewa-Speech2Text

jhdeov / interlingual-MFA

luciusssss / mc2_corpus

IgnatiusEzeani / IGBONLP

Rumeysakeskin / ASR-fine-tuning-for-low-resource-languages

clefourrier / CopperMT

harmanpreet93 / low-resource-machine-translation

fajri91 / minangNLP

ruoyuxie / noisy_parallel_data_alignment

tafseer-nayeem / BengaliReadability

alecokas / swahili-text-gcn

ofdn / OpenSpeaks-Before-AI