Transformer encoder - visualized
This repo contains the slides and the notebook
You can take a look at the notebook using NBViewer: nbviewer-link
Update: On slide 15, you don't need to set the embeddings of the masked words to all zeros, just leave it as the embedding of the MASK token. Becomes more clear in the notebook! Will update the slides soon.