mfa / allennlp-reverse-seq2seq

Sequence-to-Sequence example with AllenNLP and attention visualization

reverse

about

Train a seq2seq encoder-decoder model to reverse the input string.

rebuild of the tutorial of joeynmt: https://joeynmt.readthedocs.io/en/latest/tutorial.html in AllenNLP

Blog posts describing the code in this repository:

• https://madflex.de/posts/allennlp–reverse-sequence-example/
• https://madflex.de/posts/allennlp–sequence-to-sequence-attention-plots/

first

download generator script from joeynmt

mkdir -p tools; cd tools
wget https://raw.githubusercontent.com/joeynmt/joeynmt/master/scripts/generate_reverse_task.py
cd ..

generate some data

mkdir data; cd data
python ../tools/generate_reverse_task.py
cd ..

convert data to tsv for seq2seq datareader

cd data
paste dev.src dev.trg > dev.csv
paste test.src test.trg > test.csv
paste train.src train.trg > train.csv
cd ..

training

allennlp train configurations/reverse_starting_point.json -s output -f --include-package library

predict / evaluation

echo '{"source": "15 28 32 4", "target": "4 32 28 15"}' > reverse_example.json
allennlp predict output/model.tar.gz reverse_example.json --predictor seq2seq

predict with attention plots

allennlp predict output/model.tar.gz --use-dataset-reader examples.csv --predictor my_seq2seq --output-file output/examples.output --include-package library
python tools/plot_attention.py