The result is random

[qfmy]

Everytime I run the demo below，the result is random. Like:

first time:
Label probs: tensor([[0.1304, 0.1254, 0.7442]])
second time:
Label probs: tensor([[0.0421, 0.4105, 0.5474]])
third time:
Label probs: tensor([[0.0628, 0.8441, 0.0931]])

Is this normal？

import torch
from PIL import Image
import open_clip

model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k')
tokenizer = open_clip.get_tokenizer('ViT-B-32')

image = preprocess(Image.open("docs/CLIP.png")).unsqueeze(0)
text = tokenizer(["a diagram", "a dog", "a cat"])

with torch.no_grad(), torch.cuda.amp.autocast():
image_features = model.encode_image(image)
text_features = model.encode_text(text)
image_features /= image_features.norm(dim=-1, keepdim=True)
text_features /= text_features.norm(dim=-1, keepdim=True)

text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)

print("Label probs:", text_probs) # prints: [[1., 0., 0.]]

[Ross Wightman]

@qfmy it's not normal and it does not occur for me. Are you sure there isn't an issue on your setup loading the pretrained weights? corrupt file download?

[Ross Wightman]

If you happened to have enabled stochastic depth or dropouts, you need model.eval() after creation, just updated the README