Integrate model into existing NLP system
stefan-it opened this issue · comments
Hi @timoschick,
first, thanks for releasing the code of the form-context model ❤️
I'm currently reading the AAAI and the NAACL papers, and I'm really excited to try model on downstream tasks (like e.g. for NER).
I would like to ask, if you could provide a kind of example notebook, that shows a) how to load a trained model and b) how to get embeddings for rare words.
I don't have any experience of how attentive mimicking works on tasks like NER or PoS tagging, but I would really like to integrate such an approach into the flair
library.
Cheers,
Stefan
Hi @stefan-it ,
thanks for your interest in our work :)
Unfortunately, I do not have much time right now, so at least for the next few weeks, I will not be able to provide an example notebook. However, I have just today uploaded a new version of the form-context model that also comes with an updated Readme file which contains more detailed instructions on how to (i) train a model and (ii) get embeddings for rare words using a trained model.
Best regards
Timo
Hi Timo,
thanks for your reply! Over the holidays I'll definitely check out your implementation (great, that preprocessing is done in Python now) :)
Do you know how long the training will run on standard hardware (like an RTX card)? I would like to train a model from scratch.
Thanks,
Stefan
Hi Stefan,
Do you know how long the training will run on standard hardware (like an RTX card)?
the version used in the paper was trained using a single Nvidia GeForce GTX 1080 Ti, with which training on Wikipedia for 5 epochs took approximately 8 hours, if I remember correctly. However, some of the model's operations (in particular, the n-gram embedding lookup) are executed on CPU.
Best regards
Timo
Thanks :)
I was able to train a model from scratch and I had no problems 👍
I'll close that issue now and will open a new one as long as I have further questions :)