Multilingual Neural Machine Translation using Transformers with Conditional Normalization.
Transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.
Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example
Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb
The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.
The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice-Versa.
The Many to Many model is trained on English-French, French-English, English-German and German-English.
The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.
I also provide demos for Few-Shot and Zero-Shot translation.
Have provided SavedModels, as well as standalone notebooks that run in Google Colaboratory.
Code and Demos for the following use-cases
Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/one_to_many_demo.ipynb
Code: https://github.com/suyash/mlt/blob/master/mlt/one_to_many.py
Model: https://dl.dropboxusercontent.com/s/bssvce8t00zjwxj/one_to_many.zip?dl=0
English to French, German, Spanish and Italian translation.
This uses split vocabularies of size 8192 each.
Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_one_demo.ipynb
Code: https://github.com/suyash/mlt/blob/master/mlt/many_to_one.py
Model: https://dl.dropboxusercontent.com/s/xmroc5u6k17t5ds/many_to_one.zip?dl=0
French, German, Spanish and Italian to English translation.
This uses split vocabularies of size 8192 each.
Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_demo.ipynb
Code: https://github.com/suyash/mlt/blob/master/mlt/many_to_many.py
Model: https://dl.dropboxusercontent.com/s/kzxryp6snnx5dw6/many_to_many.zip?dl=0
English, French and German on both ends.
This uses a shared vocabulary of size 40,960.
Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb
Code: https://github.com/suyash/mlt/blob/master/mlt//many_to_many_fine_tune.py
Model: https://dl.dropboxusercontent.com/s/3ixjojbpho2emd8/many_to_many_fine_tune.zip?dl=0
Portuguese added from ted_hrlr_translate to the Many to Many model.