bplank / multilingualtokenizer

A trivial punctuation-based sentence splitter and tokenizer for multilingual data.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

bplank/multilingualtokenizer Stargazers