VITA-Group/ViT-Anti-Oversmoothing Issues
Cosine Similarity Graph
UpdatedAbout FeatScale's code
Closed 1Question about Figure 5
Closed 11FLOPs about the attention scale
Closed 3
[ICLR 2022] "Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice" by Peihao Wang, Wenqing Zheng, Tianlong Chen, Zhangyang Wang