SHI-Labs / Prompt-Free-Diffusion

Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models, arxiv 2023 / CVPR 2024

Home Page:https://arxiv.org/abs/2305.16223

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Question] Pre-computed unconditional embeddings

BlenderNeko opened this issue · comments

I noticed that the assets folder contains some pre-computed embeddings that function as the unconditional input. Were these computed similarly to how the graido app uses black images to provide the unconditional embedding?

Although we provide assets/clip_ug.pth, we didn't actually use that. The UG for prompt-free diffusion is all zeros vectors.
But prompt-free is totally fine with traditional UG (i.e. negative prompts), you may try use it. Our experience shows little improvement.

The anime app, however, assets/anime_ug.pth provided improvement in results, so we apply that in the app.