About
Single-file implementation of Attention Is All You Need
Why
- It's a flexible interpretable architecture, more performant than RNN and applicable to POMDP.
- Simple implementantion to play with my Titan V and see how much I can get out of mixed-precision learning and matmul-friendly tensor-cores with matmul-heavy architecture.