cloneofsimo / paint-with-words-sd

Implementation of Paint-with-words with Stable Diffusion : method from eDiff-I that let you generate image from text-labeled segmentation map.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Is there a way to use with other models?

hiss-remi opened this issue · comments

Just what the topic says.

Yes, certainly. I'm planning to add this feature as well. Needs major refactoring tho

If you are developer, you can replace first two argument of loaded materials to replace the model you wish and by swapping forward function. If you have "original checkpoints", (ones that are often used in a1111 repo), you can use this script to transform them into HF diffuser format.

You can now convert LDM model format (ones that are used in a1111 repos, or sd-original.ckpt formats) with paint-with-words.
Mitigated by f86c6fd

In case this gets integrated into stable-diffusion-webui, what would the preferred approach? Pre-convert the ckpt file with the script when generating something for the first time with a new model, or further rewrite the extension to support the ckpt model directly? Is the later even possible?