HazyResearch / m2

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Multilingual?

RibinMTC opened this issue · comments

Are you planning to release a multilingual version of this model? Could I finetune the current m2_bert model on german data?

We currently are not planning on training a multilingual version, but let me know if the finetuning works! It's currently using the BERT tokenizer, so I'm not sure how well that maps to German.