The result is random
qfmy opened this issue · comments
Everytime I run the demo below,the result is random. Like:
first time:
Label probs: tensor([[0.1304, 0.1254, 0.7442]])
second time:
Label probs: tensor([[0.0421, 0.4105, 0.5474]])
third time:
Label probs: tensor([[0.0628, 0.8441, 0.0931]])
Is this normal?
import torch
from PIL import Image
import open_clip
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k')
tokenizer = open_clip.get_tokenizer('ViT-B-32')
image = preprocess(Image.open("docs/CLIP.png")).unsqueeze(0)
text = tokenizer(["a diagram", "a dog", "a cat"])
with torch.no_grad(), torch.cuda.amp.autocast():
image_features = model.encode_image(image)
text_features = model.encode_text(text)
image_features /= image_features.norm(dim=-1, keepdim=True)
text_features /= text_features.norm(dim=-1, keepdim=True)
text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
print("Label probs:", text_probs) # prints: [[1., 0., 0.]]
@qfmy it's not normal and it does not occur for me. Are you sure there isn't an issue on your setup loading the pretrained weights? corrupt file download?
If you happened to have enabled stochastic depth or dropouts, you need model.eval() after creation, just updated the README