This project implements the LSTM Autoencoder for sequence modeling. The model reads a sequence and decodes itself. The model can be easily extended for any encoder-decoder task.
This code requires Torch7 and nngraph
In general, with proper parameter settings the model can recover 80%-90% of the words, when tested on a small subset of Toronto movie book corpus[http://www.cs.toronto.edu/~mbweb/].
To train a model with default setting, simply run th LSTMAutoencoder.lua The code generates samples at validation time, to inspect the effective of reconstruction. One may consider to use the Autoencoder to obtain general purpose sentence vectors, or as a pretraining step for downstream tasks
-
Li, Jiwei, Minh-Thang Luong, and Dan Jurafsky, "A hierarchical neural autoencoder for paragraphs and documents", arXiv preprint arXiv:1506.01057 (2015).
-
Dai, Andrew M., and Quoc V. Le, "Semi-supervised sequence learning", Advances in Neural Information Processing Systems. 2015.