Experiments on Variations of Neural Machine Translation System
A Detailed Analysis for Vietnamese to English and Chinese to English Translation
Tasks:
- Recurrent neural network based encoder-decoder without attention
- Recurrent neural network based encoder-decoder with attention
- Replace the recurrent encoder with either convolutional or self-attention based encoder
- Build either or both fully self-attention translation system
To Run:
RNN based encoder-decoder without attention
python main.py --language zh --save_model_name zh_rnn --FT_emb_path ft_emb \\
--data_path MT_data --goal zh_rnn --decoder_type basic
RNN based encoder-decoder with attention
python main.py --language zh --save_model_name zh_attn --FT_emb_path ft_emb \\
--data_path MT_data --goal zh_attn --decoder_type attn
Self-attention based encoder and RNN based decoder with attention
python main.py --language zh --save_model_name zh_attn --FT_emb_path ft_emb \\
--data_path MT_data -goal zh_selfattn --self_attn True
Transformer
python main.py --language zh --save_model_name zh_transformer --FT_emb_path ft_emb \\
--data_path MT_data -goal zh_transformer --transformer True
Test
python main.py --language zh --save_model_name zh_attn --FT_emb_path ft_emb \\
--data_path MT_data -goal zh_transformer --test_only True
Experiments Results:
Sacre-BLEU scores of three models on test set
Model | Vietnamese(vi) | Chinese(zh) |
---|---|---|
Baseline | 8.19 | 7.23 |
RNN + Attention | 17.69 | 12.83 |
Self-attention + Attention | 13.27 | 7.97 |
LSTM vs GRU
Model | Vietnamese(vi) | Chinese(zh) |
---|---|---|
GRU | 16.37 | 11.86 |
LSTM | 17.06 | 11.94 |
Sample Tranaslation Results:
Chinese to English
Source Sentence: 对 于 阿 富 汗 , 我 意 识 到 — — 这 是 西 方 国 家 也 不 曾 意 识 到 的 — — 即 在 我 们 成 功 的 背 后 是 一 个 这 样 的 父 亲 — — 他 能 认 识 到 自 己 女 儿 的 价 值 , 也 明 白 女 儿 的 成 功 , 就 是 他 自 己 的 成 功 。
Target Sentence: what i ve come to realize about afghanistan , and this is something that is often dismissed in the west , that behind most of us who succeed is a father who recognizes the value in his daughter and who sees that her success is his success
Translation: and in fact , i realize that this is nothing in the west that is the father of the end of the world who can recognize his daughter s value , and it s also his daughter success , is his own
Vietnamese to English
Source Sentence: tôi đã khôngtưởng được những gì xảy đến với cuộcsống của một người tịnạn từ Bắc UNK thì sẽ như thếnào nhưng tôi sớm nhận ra rằng nó khôngnhững rất khókhăn màcòn vôcùng nguyhiểm vì những người tịnạn từ Bắc UNK vào trungQuốc đều bị coi là dân nhậpcư tráiphép
Target Sentence: i had no idea what life was going to be like as a north korean refugee , but i soon learned it s not only extremely difficult , it s also very dangerous , since north korean refugees are considered in china as illegal migrants
Translation: i was longing for what happened to the life of refugees in northern ireland , and i would find out that it wasn t difficult to be very , but dangerous because refugees from north korea is not allowed by bullies
References:
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
- Neural Machine Translation by Jointly Learning to Align and Translate
- Attention is all you need
- A decomposable attention model
- Training Deeper Neural Machine Translation Models with Transparent Attention
- Effective Approaches to Attention-based Neural Machine Translation
- Harvardnlp, The Annotated Transformer