Why Remove PreNorm?
tonyyunyang opened this issue · comments
May I ask why the PreNorm is removed? I am very curious about the reason.
As the Transformer encoder is different from the architecture as shown below, which is from the original paper.
lucidrains / vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch