Unofficial implementation algorithms of attention models on SNLI dataset.
Current include papers:
- "Reasoning About Entailment With Neural Attention", arXiv:1509.06664
- "Learning Natural Language Inference with LSTM", arXiv:1512.08849
Based on Lasagne.
- Python 3
- Lasagne
- some others(best install Anaconda)
At source root dir
First extracts preprocessed SNLI data
./extract_data.sh
Then run:
python3 ./snli_reasoning_attention.py [condition|attention|word_by_word]
Or run:
python3 ./snli_match_lstm.py
The learning curve of word by word attention:
The learning curve of match LSTM with word embedding:
About word by word attention:
- The test acc of match LSTM is 1% smaller than the original paper, 82.31%
About match LSTM:
- The test acc of match LSTM is 1% smaller than the original paper.
- not decay learning rate when train
- using Word2Vec pre-trained model
- tune OOV words in train dataset, like in
Reasoning Attention
[3]: