BenJamesbabala / neuralHMM

end to end unsupervised learning Neural Hidden Markov Model

neuralHMM

Code of end to end unsupervised neural HMM. Please make a PR if you want to contribute to the code.

Unsupervised Neural Hidden Markov Models  
Ke Tran, Yonatan Bisk, Ashish Vaswani, Daniel Marcu, and Kevin Knight   
EMNLP 2016,  Workshop Structured Prediction for Natural Language Processing

Requirement

To run the code, you need

• Torch
• lua-utf8, which can be install easily luarocks install luautf8
• cudnn.torch

Note

• You can enjoy some speed up if you use cudnn.LSTM instead. However, we see better performance when using Justin's LSTM implementation. This is probably due to parameter initialization, which is sensitive in unsupervised learning.

Data preprocessing

• tokenized, replace all digit by 0 (e.g tr '0-9' 0)
• DO NOT lowercase
• one sentence per line

Acknowledgment

The code is ultilized from

• Justin Johnson's torch-rnn
• Yoon Kim's char-lstm