Model as input to ACGD Optimizer rather than parameters
jshanna100 opened this issue · comments
Is there a reason why the ACGD Optimizer takes entire models as input rather than Parameter lists? This is different from general Pytorch optimizer usage, as well as the other CGD optimizers. This makes usage inconvenient where the Discriminator/Generator are made of more than one network.
Oh, good point! Technically there is no difficulty to make ACGD's input as same as general optimizer's. I'm planning to update this. Thanks for your advice.
May I ask, has this point been realized?
Oh, good point! Technically there is no difficulty to make ACGD's input as same as general optimizer's. I'm planning to update this. Thanks for your advice.
Has it been realized now? Brothers
Yes, the code has been updated. Please checkout this CGDs packages https://github.com/devzhk/cgds-package if you are interested.
You can just wrap them into one nn.Module.
Not sure what you mean by using this ddp. Do you mean using acgd with Pytorch DistributedDataParallel?
Sorry, I don't understand your question. If you want to use ACGD with Pytorch DDP, please see the instructions in the README of https://github.com/devzhk/cgds-package.
You can always wrap two networks into one torch.nn.Module.
For example,
class MetaModel(torch.nn.Module):
def __init__(net1, net2):
self.net1 = net1
self.net2 = net2
def forward(net1_input, net2_input):
out1 = net1(net1_input)
out2 = net2(net2_input)
return out1, out2ACGD can accept this MetaModel. In principle, you can always wrap all the networks with the same optimization direction into one MetaModel and feed it to the optimizer.
Do you mean the problem where you have two generators? Nope, I haven't but it should be straightforward to get around.
I can't see any code or error in your message. They are not displayed in your comment.
Hi, would it be possible for you to share the code directly on Github issue? This way, it can be displayed in the comment and it would be much easier for me to read and follow along this issue.