huawei-noah / Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

wukong模型图文测试相似度太低

douzi0248 opened this issue · comments

commented

使用图文对进行测试,模型是vit_b模型进行测试,图文相似度[0.0937,0.0846,0.0857]

您好,这个结果是logits,未做softmax吧,如果想根据logits计算图文相似度,建议follow CLIP工作的方式,* temperature parameter以后再softmax:

logits = (100 * image_features @ text_features.T).softmax(dim=-1)