Degenerate issue when conditioned by an image
BoltenWang-Meta opened this issue · comments
Thx for impressive work and code. So, after checking the inference code under the case conditioned by an image, the whole generation just degenerates into a SSL representation conditioned MAGE. Is my understanding true? If yes, does it mean RDM is not used during inference?
Thanks for your interest! For Figure 6 and 7 in the paper (which corresponds to the case conditioned by ground-truth images), the RDM is NOT needed because the SSL representation is provided by ground-truth image. However, we note that such a case has a strong limitation in practice, as typically we don't have ground-truth images when we want to generate an image. Therefore, we need the RDM to generate the SSL representations under common generation scenarios where we don't have ground-truth images
Got it! I agree with generating images conditioned by GT seems to be useless. I was just wondering where does RDM go in viewing your code. Plus, thx for so quick reply, what a dedicated author you are. hhhh
We integrate the RDM sampling process in the pixel generator. For example, you can check it here in the MAGE generator: https://github.com/LTH14/rcg/blob/main/pixel_generator/mage/models_mage.py#L485-L508. I just happened to see this issue pop out -- hope my response can solve your questions.