Questions about extending the first convolutional layer

[wuyujack (Mingfu Liang)]

Congrats on your work! In the paper, you mentioned that:

we propose to extend the kernel channels of the first convolutional layer by adding zero initialized weights to match the new input channel dimension

Will you also fine-tune the first convolutional layer or the stable diffusion model during your training to accommodate for the channel change?

BTW, will the code be released before the end of June?

[Alberto Baldrati]

Hi @wuyujack
Thanks for your interest in our work!

To adapt the network to the extended kernel channels we train all the stable diffusion model weights during the training.
We plan to release the inference code before the end of June.

Alberto