Prompt Fusion is an auto1111 webui extension that adds more possibilities to the native prompt syntax. Among other additions, it allows to interpolate between the embeddings of different prompts, continuously:
# linear prompt interpolation
[night light:magical forest: 5, 15]
# catmull-rom curve prompt interpolation
[night light:magical forest:norvegian territory: 5, 15, 25:catmull]
# linear attention interpolation
(fire extinguisher: 1.0, 2.0)
# prompt-editing-aware attention interpolation
[(fire extinguisher: 1.0, 2.0)::5]
The prompt interpolation feature is similar to Prompt Travel, which allows to create videos of images generated by navigating the latent space iteratively. Unlike Prompt Travel however, instead of generating multiple images, Prompt Fusion allows you to travel during the sampling process of a single image. Also, instead of interpolating the latent space, it uses the embedding space to determine intermediate embeddings.
Prompt interpolation is also similar to Prompt Blending. The main difference is that this extension calculates a new embedding for every step, as opposed to calculating it once and using that same one embedding for all the steps.
The attention interpolation feature is similar to Shift Attention, which allows to generate multiple images with slight variations in the attention given to certain parts of the prompt. Unlike Shift Attention, instead of generating multiple images, Prompt Fusion allows to shift the attention of certain parts of a prompt during the sampling process of a single image.
Interpolate linearly (by default) from lion (step 0) to bird (step 8) to girl (step 11), and stay at girl for the rest of the sampling steps:
[lion:bird:girl: , 7, 10]
Interpolate using a bezier curve from fireball monster (step 0) to dragon monster (step 12, because 30 steps * 0.4 = step 12), while using seawater monster as an intermediate control point to steer the curve away during interpolation and to get creative results:
[fireball:seawater:dragon: , .1, .4:bezier] monster
- Prompt interpolation using a curve function
- Attention interpolation aware of contextual prompt editing
- Complete backwards compatibility with prompt syntax from https://github.com/AUTOMATIC1111/stable-diffusion-webui
The following webui releases are officially supported:
v1.0.0-pre
- Visit the Extensions tab of Automatic's WebUI.
- Visit the Install from URL subtab.
- Copy and paste
https://github.com/ljleb/prompt-fusion-extensionin the URL for extension's git repository textbox. - Press the Install button.
- Check the wiki pages for the extension documentation.