Why divide by temp when calculating cosine similarity

[zqiang]

`class Similarity(nn.Module):
"""
Dot product or cosine similarity
"""

def __init__(self, temp):
    super().__init__()
    self.temp = temp
    self.cos = nn.CosineSimilarity(dim=-1)

def forward(self, x, y):
    return self.cos(x, y) / self.temp`

[Tianyu Gao]

Hi, please check our paper for details. Temperature is one of our hyperparameters and we use it to adjust how "sharp" the distribution is.