- Patch partition
- Patch merging
- Relative position bias
- Feature map padding
- Self-attention in non-overlapped windows
- Shifted Window based Self-Attention
PyTorch implementation of 'Swin Transformer' (Liu et al., 2021) from scratch.
PyTorch implementation of 'Swin Transformer' (Liu et al., 2021) from scratch.