Using compel affects SDXL 0.9 output (degraded results)

[Rehan H]

I've been tracking the pr of supporting SDXL 0.9 and saw that released a few hours ago. I've tested and based on my results, I can say that when compel is used with SDXL, the output is degraded in quality without even using and weighting.

The example code was taken from: https://pypi.org/project/compel/2.0.0/

PS: I had to omit returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED in order to make it work. Not sure where to import ReturnedEmbeddingsType

import torch
from compel import Compel
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-0.9", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
pipe.to("cuda")

compel = Compel(tokenizer=[pipe.tokenizer, pipe.tokenizer_2] , text_encoder=[pipe.text_encoder, pipe.text_encoder_2], requires_pooled=[False, True])

# Without compel
generator = torch.Generator(device="cuda").manual_seed(40)
prompt = "Shot of Vaporwave fashion dog in miami, cinematic"

images = pipe(prompt=prompt, generator=generator, guidance_scale=7.5, num_images_per_prompt=1, num_inference_steps=50).images;

for image in images:
  image.show()

# With compel
generator = torch.Generator(device="cuda").manual_seed(40)

prompt = "Shot of Vaporwave fashion dog in miami, cinematic"
conditioning, pooled = compel(prompt)

images = pipe(prompt_embeds=conditioning, pooled_prompt_embeds=pooled, generator=generator, guidance_scale=7.5, num_images_per_prompt=1, num_inference_steps=50).images;

for image in images:
  image.show()

Without Compel Results:

With Compel Results:

[Damian Stewart]

I had to omit returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED in order to make it work

omitting that will indeed result in degraded output.

try from compel import Compel, ReturnedEmbeddingsType

[Damian Stewart]

i've updated the README, please LMK if the import above resolves your issue

[Rehan H]

Yep, that solved it. Will be testing further with weighing etc. Closing for now.