Train on a custom dataset?

Question

Train on a custom dataset?

alelordelo opened this issue a year ago · comments

Alexandre Lordelo commented a year ago

Hi, thanks for the amazing paper and code! : )

Would it be possible to train a a custom dataset with segmentation maps and nested JSON prompts?

cheers

Songwei Ge · Answer 1 · Wed Apr 19 2023 17:34:30 GMT+0800 (China Standard Time)

Hi, thank you for your interest! Our method does not require any training. If you have another model, say stable diffusion finetuned on the custom dataset, you may directly load the model checkpoint and use our rich-text formats to control the generation.

Alexandre Lordelo · Answer 2 · Wed Apr 19 2023 17:50:54 GMT+0800 (China Standard Time)

thanks @songweige
Great, will try to run with with a custom model!

Does it work with image to image as well? Would be super helpful to specify prompts per masked region

Songwei Ge · Answer 3 · Thu Apr 20 2023 11:41:38 GMT+0800 (China Standard Time)

For image-to-image models, I think it would be possible to integrate with models like ControlNet. I don't think it is compatible with models like InstructPix2Pix.

Alexandre Lordelo · Answer 4 · Thu Apr 20 2023 15:30:55 GMT+0800 (China Standard Time)

What about the default stable diffusion 1.5 image to image?Sent from my iPhoneOn 20 Apr 2023, at 05:41, SongweiGe ***@***.***> wrote: For image-to-image models, I think it would be possible to integrate with models like ControlNet. I don't think it is compatible with models like InstructPix2Pix. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

Songwei Ge · Answer 5 · Sat Aug 12 2023 00:26:42 GMT+0800 (China Standard Time)

Sorry for missing your question. I think it is compatible with the SDEdit kind of method.