Hello everyone!
This repo is for a personal project in which I intend to report my path to learn Natural Language Processing (NLP) with Deep Learning.
// TODO
-
21/12/2020 to 04/01/2021 - In the last two weeks, I finished watching the first lecture of Stanford CS224N - NLP with Deep Learning - Winter 2019, which I had started the week before. Although it was a good lecture, I found it hard to follow the math of the main topic: Word2Vec. Therefore, I decided to rewatch Andrew Ng's videos on the subject, which are available on his Sequence Models course on Coursera. Andrew's videos on the subject are easier to understand and directly cite the related papers, which I found extremely helpful. I also decided to read some online hands-on tutorials. The Pierre Megret tutorial on Kaggle was very helpful to comprehend the gensim library, while Ivan Chen's Medium post was very handy to get a more in-depth and low-level understanding of the Word2Vec. However, I missed a tutorial on the subject using PyTorch. I hope to find a good one next week or create one of my own. Finally, I would like to mention that I finished the CS224 first assignment, and it was quite interesting and easy to follow. Now I expect to go faster on my studies because the last two weeks were not very productive.
-
04/01/2021 to 18/01/2021 - In the last two weeks, mainly in the first, I followed the "NLP from Scratch" tutorials available on PyTorch's official website. They were quite good, and I manually typed all the tutorial codes to absorb as much knowledge as possible. However, this was a rather time-consuming exercise. Therefore, I intend from now on to read more code and type less. I made some changes in the code, trying to clean it a little bit during tye typing. But this had more to do with Python good practices than Deep Learning per se. I noticed that the RNN training was rather slow, even for a simple network. I wondered if this can be easily solved. However, I didn't dive too much into the topic because I had a lot of Ph.D. stuff to do. In the next two weeks, I hope to study more about NLP with Deep Learning and watch more CS224N lectures (I watched only one class in the last two weeks). P.S.: This piece of text was written entirely on a single day, rather than continuously for two weeks. This compromised this public report.