Compel
A text prompt weighting and blending library for transformers-type text embedding systems, by @damian0815.
With a flexible and intuitive syntax, you can re-weight different parts of a prompt string and thus re-weight the different parts of the embeddning tensor produced from the string.
Tested and developed against Hugging Face's StableDiffusionPipeline but it should work with any diffusers-based system that uses an Tokenizer and a Text Encoder of some kind.
Adapted from the InvokeAI prompting code (also by @damian0815). For now, the syntax is fully documented here - note however that cross-attention control .swap() is currently ignored by Compel.
Installation
pip install compel
Demo
Quickstart
with Hugging Face diffusers >=0.12:
from diffusers import StableDiffusionPipeline
from compel import Compel
pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)
# upweight "ball"
prompt = "a cat playing with a ball++ in the forest"
conditioning = compel.build_conditioning_tensor(prompt)
# generate image
images = pipeline(prompt_embeds=conditioning, num_inference_steps=20).images
images[0].save("image.jpg")For batched input, use torch.cat to merge multiple conditioning tensors into one:
import torch
from diffusers import StableDiffusionPipeline
from compel import Compel
pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)
prompts = ["a cat playing with a ball++ in the forest", "a dog playing with a ball in the forest"]
prompt_embeds = torch.cat([compel.build_conditioning_tensor(prompt) for prompt in prompts])
images = pipeline(prompt_embeds=prompt_embeds).images
images[0].save("image0.jpg")
images[1].save("image1.jpg")