No difference when using consistencydecoder in diffusers

[howardgriffin]

Here is my code, however, there seems no difference between gan_vae and cosistency_vae. What's wrong?

import torch
from diffusers import StableDiffusionPipeline, ConsistencyDecoderVAE
vae = ConsistencyDecoderVAE.from_pretrained("openai/consistency-decoder", torch_dtype=torch.float16)
pipe_gan = StableDiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16 ).to("cuda")
pipe_consistency = StableDiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", vae=vae, torch_dtype=torch.float16).to("cuda")
image_gan = pipe_gan("a girl hold flower", generator=torch.manual_seed(0)).images[0]
image_consistency = pipe_consistency("a girl hold flower", generator=torch.manual_seed(0)).images[0]

[Edoardo Botta]

The output images in your example are actually slightly different. Checking how many entries match between the two tensors:

import torchvision.transforms as transforms

transform = transforms.Compose([ 
    transforms.PILToTensor() 
]) 

equals = transform(image_gan) == transform(image_consistency)
equals.sum() / torch.numel(equals)

returns

tensor(0.1281)