jacopofar / grammar-quiz

Online cloze deletion tool focused on grammar

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Do not invent spaces

jacopofar opened this issue · comments

In languages without spaces like Chinese, the card appears with spaces.
It's a problem either of tokenization or visualization (see also #28 which shares the problem of needing the language metadata reach the frontend in 3-letter ISO format and encode some behavior for specific language lists)

Solved by letting the ICU tokenizer generate the tokens in a reversible way, removed the space logic from the frontend