Bitextor Team

bitextor

Bitextor generates translation memories from multilingual websites

bicleaner

Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.

pdf-extract

PDF parser and converter to HTML

bicleaner-ai

Bicleaner fork that uses neural networks

bifixer

Tool to fix bitexts and tag near-duplicates for removal

warc2text

Extracts plain text, language identification and more metadata from WARC records

biroamer

Utility that will help you to ROAM (Random Omit Anonymize and Mix) your parallel corpus.

bleualign-cpp

neural-document-aligner

Document aligner which uses neural technologies to search matches across bilingual documents

bicleaner-data

Repository for data models, dictionaries and more resources for Bicleaner

monocleaner

monotextor

python-pdfextract

Python interface to pdf-extract, HTML extraction from PDF

bicleaner-ai-data

Repository of Bicleaner AI models

bicleaner-hardrules

Pre-filtering step for bicleaner

bitextor-data

Repository for data models, dictionaries and more resources for Bitextor

bitextor-neural

Bitextor Neural generates translation memories from multilingual websites using state-of-the-art Machine Learning tools

prevertical2text

Extracts plain text, language identification and more metadata from Spiderling prevertical files

vecalign

Improved Sentence Alignment in Linear Time and Space

loomchild-segment-py

Python module to interface with Java Loomchild sentence segmenter

monocleaner-data

Monocleaner models repository

bicleaner-ai-glove

Fork of glove-python to distribute binary builds

bitextor-testing-output

Repository for storing testing outputs from Bitextor

cld2

Compact Language Detector 2

deferred-crawling

Reconstructs sentences using deferred crawling standoff annotations from Bitextor

fastText

Library for fast text representation and classification.

hunalign

Sentence aligner

python-apachetika

Python interface to Apache Tika, HTML extraction from PDF