Tutorials for the using of Transformer layer

Question

Tutorials for the using of Transformer layer

zwx8981 opened this issue 5 years ago · comments

Hi, is there any plan to provide a tutorial of showing an example of employing the Transformer as an alternative of RNN for seq2seq task such as machine translation?

Md Hasan · Answer 1 · Sat Jul 20 2019 02:02:08 GMT+0800 (China Standard Time)

I found this great tutorial to learn about Transformers. Link: (http://jalammar.github.io/illustrated-transformer/)

If you decided to implement in PyTorch than this link is going to help you.
(http://nlp.seas.harvard.edu/2018/04/03/attention.html)

Arushi Rai · Answer 2 · Wed Jan 24 2024 01:50:07 GMT+0800 (China Standard Time)

Coming back to this! It would be cool to see transformers for video data and the categories suggested above... I could work on this if this hasn't been done. The links that have been shared were great for their time, but now there are Transformer-specific modules in Pytorch that would be good to demonstrate.