Question about transformer.norm

[Myungbin]

Hello! Thank you for sharing the great code. I have a question. Could you please explain why you applied 'transformer.norm' in the 'forward' function of 'DinoVisionTransformerClassifier'?

def forward(self, x):
    x = self.transformer(x)
    x = self.transformer.norm(x)
    x = self.classifier(x)
    return x

[purnasai]

Hi @Myungbin, Thanks for finding the repo useful. I am not externally applying "Normalization" to the features. I am using the "Normalization" layer that is inbuilt in the model architecture to "Normalize" the features.

This step adds stabilization/scales the values to be in a range. It might work even without this step, but this step has improved the Overall results. hence added.

Thank you

[Myungbin]

@purnasai Thank you very much! It was very helpful!