Bitextor Team

Bitextor Team's repositories

bitextor

Bitextor generates translation memories from multilingual websites

Language:PythonGPL-3.0291 29 160

bicleaner

Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.

Language:PythonGPL-3.0155 13 53

pdf-extract

PDF parser and converter to HTML

Language:JavaGPL-3.085 16 51

bicleaner-ai

Bicleaner fork that uses neural networks

Language:PythonGPL-3.039 11 20

bifixer

Tool to fix bitexts and tag near-duplicates for removal

Language:PythonGPL-3.030 7 11

warc2text

Extracts plain text, language identification and more metadata from WARC records

Language:C++MIT21 9 26

biroamer

Utility that will help you to ROAM (Random Omit Anonymize and Mix) your parallel corpus.

Language:PythonGPL-3.010 9 4

bleualign-cpp

Language:C++GPL-3.09 11 4

monocleaner

Language:PythonGPL-3.07 9 3

neural-document-aligner

Document aligner which uses neural technologies to search matches across bilingual documents

Language:PythonGPL-3.07 8 1

bicleaner-data

Repository for data models, dictionaries and more resources for Bicleaner

GPL-3.06 10 1

bitextor-data

Repository for data models, dictionaries and more resources for Bitextor

GPL-3.06 90

monotextor

Language:PythonGPL-3.06 10 1

python-pdfextract

Python interface to pdf-extract, HTML extraction from PDF

Language:PythonNOASSERTION6 10

bicleaner-ai-data

Repository of Bicleaner AI models

NOASSERTION5 90

bicleaner-hardrules

Pre-filtering step for bicleaner

Language:PythonGPL-3.04 8 4

bitextor-neural

Bitextor Neural generates translation memories from multilingual websites using state-of-the-art Machine Learning tools

Language:PythonGPL-3.03 90

prevertical2text

Extracts plain text, language identification and more metadata from Spiderling prevertical files

Language:C++MIT2 8 1

vecalign

Improved Sentence Alignment in Linear Time and Space

Language:PythonApache-2.02 30

loomchild-segment-py

Python module to interface with Java Loomchild sentence segmenter

Language:PythonGPL-3.01 2 1

monocleaner-data

Monocleaner models repository

GPL-3.01 80

scrawl

Playwright-based web crawler

Language:PythonGPL-3.01 70

bicleaner-ai-glove

Fork of glove-python to distribute binary builds

Language:PythonApache-2.0010

bitextor-testing-output

Repository for storing testing outputs from Bitextor

GPL-3.0080

cld2

Compact Language Detector 2

Language:C++Apache-2.0030

deferred-crawling

Reconstructs sentences using deferred crawling standoff annotations from Bitextor

Language:PythonMIT080

fastText

Library for fast text representation and classification.

Language:HTMLMIT010

hunalign

Sentence aligner

Language:C++LGPL-3.0020

python-apachetika

Python interface to Apache Tika, HTML extraction from PDF

Language:PythonNOASSERTION010