parallel-corpus

There are 2 repositories under parallel-corpus topic.

• NiuTrans / Classical-Modern

  非常全的文言文（古文）-现代文平行语料

  corpusparallel-corpustraditional-and-simplified-chinesetraditional-chinese
  Language:Python 921

• kirralabs / indonesian-NLP-resources

  data resource untuk NLP bahasa indonesia

  datasetnlpcorpus-linguisticscorpusindonesian-languageindonesiannamed-entity-recognitionpos-taggingdependency-parsercrawlersentiment-analysisparallel-corpus
  220

• csebuetnlp / banglanmt

  This repository contains the code and data of the paper titled "Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation" published in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), November 16 - November 20, 2020.

  bangla-nlpmachine-translationparallel-corpusparallel-corporaneural-machine-translationbangla-dataset-machine-translationbangla-machine-translationlow-resource-languagesemnlp-2020low-resource-nlplow-resource-machine-translation
  Language:Python 145

• Helsinki-NLP / OpusFilter

  OpusFilter - Parallel corpus processing toolkit

  corpus-toolscorpus-processingnatural-language-processingmachine-translationparallel-corpusnlp
  Language:Python 96

• bfsujason / bertalign

  Multilingual sentence alignment using sentence embeddings

  sentence-alignmentbertparallel-corpustranslation-memorycross-lingual-embeddings
  Language:Python 86

• tsuruoka-lab / BSD

  The Business Scene Dialogue corpus

  corpusmachine-translationjapaneseenglishdocument-alignedannotated-corporaparallel-corporaparallel-corpus
  65

• FerreroJeremy / Cross-Language-Dataset

  A multilingual, multi-style and multi-granularity dataset for cross-language textual similarity detection

  datasetmulti-granularity-datasetcross-language-datasetmulti-language-datasetparallel-corpus
  60

• matbahasa / TALPCo

  TUFS Asian Language Parallel Corpus

  parallel-corpusjapaneseburmesemyanmarmalaybahasa-melayuindonesianbahasa-indonesiaenglishvietnamesetiengvietthaiinterpersonalmeaningtokenized-sentencesaddresseekoreantreebankconstituency-treejavanese
  Language:TeX 48

• ShathaTm / LK-Hadith-Corpus

  Leeds University and King Saud University (LK) Hadith Corpus

  hadithdatasetparallel-corpuscorpusarabic-englishhadeeth
  Language:Python 48

• sharad461 / nepali-translator

  Neural Machine Translation on the Nepali-English language pair

  machine-translationnepali-englishparallel-corpusdata-cleaning
  Language:Python 46

• asmelashteka / HornMT

  Machine translation (MT) benchmark dataset for languages in the Horn of Africa.

  machine-translationnatural-language-processingdatasethorn-of-africaparallel-corpusethiopiaafaramharicoromosomalitigrinyaafricamachine-learning
  37

• Kartikaggarwal98 / Indian_ParallelCorpus

  Curated list of publicly available parallel corpus for Indian Languages

  machinetranslationindian-languagesnlpcorpusparallel-corpusparallel-corporamultilingual-translationlow-resource-machine-translationlow-resource-languagesneural-machine-translation
  28

• danielinux7 / Abkhaz-NLP-Data-Pipeline

  Abkhazian language focused multilingual and monolingual corpuses for Natural Language Processing(NLP)

  neural-machine-translationparallel-corpus
  Language:Python 18

• BramVanroy / astred

  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For instance useful for comparing a translation with the original text, to find differences and similarities between two different translations, or to see how a machine translation differs from a reference translation.

  translationparallel-corpusalignmentlinguisticsnlpparsingspacystanza
  Language:Python 17

• cfiltnlp / IITB-English-Hindi-PC

  The IIT Bombay English-Hindi Parallel Corpus

  machine-translationparallel-corpuscorpus
  Language:Jupyter Notebook 16

• priyanshu2103 / Sanskrit-Hindi-Machine-Translation

  Machine Translation from Sanskrit to Hindi using Unsupervised and Supervised Learning

  hindimachine-translationsanskritparallel-corpusmonolingual-corporasanskrit-englishfasttext-embeddings
  Language:Jupyter Notebook 16

• korenyoni / opus-api

  OPUS (opus.nlpl.eu) Python3 API

  pythonmachine-learningopusapilanguage-modelparallel-corpusparallel-corporacorporatecorporacorpus
  Language:Python 14
• IESTAC

  Giuseppe-Della-Corte / IESTAC

  A corpus that can be used to train English-to-Italian End-to-End Speech-to-Text Machine Translation models

  machine-translationspeech-translationcorpusparallel-corpusparallel-corporaend-to-end-machine-learningforced-alignmentspeech-processingmfcc-featuresbitextsentence-embeddingssentence-similaritystatistical-machine-translationspeech-recognitiontext-processingtext-preprocessinigweb-scrapingnamed-entity-recognitionaudio-datasql-database
  11
• PARASITE

  YerevaNN / PARASITE

  🪱 PARASITE || A parallel sentence data preprocessing toolkit. Originally developed as a part of the `en-ru` winner submission of WMT20 Biomedical Translation Task.

  machine-translationpreprocessingalignmentsparallel-corpusnatural-language-processingdeep-learningcrosslingual
  Language:Python 11

• farshadjafari / parallel_corpus_generator

  Python application, generating parallel corpus for any language pairs, can be used for training nmt (Neural Machine Translation) systems

  computational-linguisticsnatural-language-processingneural-machine-translationparallel-corpusstatistical-machine-translation
  Language:Python 10

• soumendrak / MTEnglish2Odia

  Machine Translation from English to Odia language.

  machine-translationodia-languageodiaindic-languagesparallel-corpuspython3
  Language:Jupyter Notebook 9

• spraakbanken / swell-editor

  Editor for normalising learner texts (error annotation and tagging.)

  slasecond-language-acquisitionparallel-corpusannotation-toolswell-editorswell
  Language:TypeScript 9

• tsuruoka-lab / AMI-Meeting-Parallel-Corpus

  AMI Meeting Parallel Corpus

  corpusmachine-translationenglishjapaneseparallel-corpusparallel-corporadocument-alignedannotated-corpora
  9

• x39826 / Pali_Tripitaka

  Pali Buddhist scriptures of 15 countries and its parallel corpus

  pali-tripitakaparallel-corpus
  Language:Python 9

• michmech / irish-sentence-bank

  4,500 sentences in Irish, tokenized, manually lemmatized, translated into English.

  lemmatizationparallel-corpusgaeilgeirish
  8

• shashanksiripragada / pib-crawl

  Code to extract multilingual parallel corpus from Press Information Bureau (PIB) website.

  parallel-corpusindian-languageneural-machine-translationmultilingual-nmt
  Language:Python 8

• KurdishBLARK / InterdialectCorpus

  A parallel corpus of Sorani, Kurmanji and English

  kurdishparallel-corpusnatural-language-processingkurdish-language-processingcorpusmachine-translation
  7

• UUDigitalHumanitieslab / timealign

  Parallel corpus annotation and visualization

  parallel-corpusannotationvisualizationdjango-application
  Language:Python 7

• x39826 / Multilang_Translator_For_Pali_Tripitaka

  Parallel corpus and multilingual machine translation system of the Pali Buddhist scriptures in 15 countries（15国巴利文大藏经平行语料与多语言机器翻译系统）

  pali-tripitakamachine-translationparallel-corpus
  Language:Python 7

• stibiumghost / tajik-to-persian-transliteration

  Tajik-to-Persian transliteration model

  farsiparallel-corpuspersianseq2seqtajiktransliteration
  Language:Jupyter Notebook 6

• UUDigitalHumanitieslab / perfectextractor

  Extracting present perfects (and related forms) from parallel corpora

  parallel-corpusextractionxpath
  Language:Python 6

• PyThaiNLP / Thai-Lao-Parallel-Corpus

  Thai Lao Parallel corpus

  thai-languagelao-languagecorpusparallel-corpus
  5

• tanloong / interlaced.nvim

  Neovim plugin for aligning bilingual parallel texts

  nvim-pluginparallel-corpussentence-alignmentcorpus-linguistics
  Language:Lua 5
• suoyan.pro

  TienZhao / suoyan.pro

  Online parallel text alignment tool.

  corpusparallel-corpustext-alignmentcorpus-tools
  Language:TypeScript 5

• OdiaWikimedia / English-Odia

  English to Odia/Oriya parallel corpus of phrases

  odia-languageodiaparallel-corpusenglish-translation
  Language:Jupyter Notebook 4

• rggdmonk / hadal

  A simple and efficient tool for mining and aligning sentences with pre-trained models.

  alignmentnlpnlp-libraryparallel-corporaparallel-corpussentence-alignmentparallel-sentence-miningmachine-translation
  Language:Python 4