frank-xwang / InstanceDiffusion

[CVPR 2024] Code release for "InstanceDiffusion: Instance-level Control for Image Generation"

Home Page:https://people.eecs.berkeley.edu/~xdwang/projects/InstDiff/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About Iterative Image Generation

Chaoriticle opened this issue · comments

I have tried the Iterative Image Generation demo, like this
image
image
But the picture I got is so different from your demo.The main problem is the result of r2 is not generate based on the r1, I dont know if I did something wrong.
r1:
Snipaste_2024-05-19_21-17-18
r2:
Snipaste_2024-05-19_21-17-56

Hello! Indeed, the difference between R1 and R2 is the largest as it involves adding two medium/large objects in a single run. To improve the model’s adherence to instance-level control signals, you might consider increasing the values of 'mis' and 'alpha'. Or simply try a few more seeds :-) !
Please be aware that our model is primarily designed for image generation, not image editing. The capability to iteratively generate images is essentially a bonus feature, made possible by the residual connection design of our UniFusion module. If your goal is to find the best method to generate a new image based on an existing one with minimal differences, you may want to fine-tune our model for this task by adding an image as additional inputs, or explore image editing methods, which are specifically tailored for such tasks.

Attached results are 4 demos images I got:

image

Hope it helps. :-)

Hello! Indeed, the difference between R1 and R2 is the largest as it involves adding two medium/large objects in a single run. To improve the model’s adherence to instance-level control signals, you might consider increasing the values of 'mis' and 'alpha'. Or simply try a few more seeds :-) ! Please be aware that our model is primarily designed for image generation, not image editing. The capability to iteratively generate images is essentially a bonus feature, made possible by the residual connection design of our UniFusion module. If your goal is to find the best method to generate a new image based on an existing one with minimal differences, you may want to fine-tune our model for this task by adding an image as additional inputs, or explore image editing methods, which are specifically tailored for such tasks.

Attached results are 4 demos images I got:

image Hope it helps. :-)

Thanks a lot! Grateful for your support! ♪(・ω・)ノ