How to build a GAN with O-CNN ?
TrepangCat opened this issue · comments
Hi, I am trying to build a GAN network with your O-CNN code. Here are some problems I want to ask:
- During the forward process of convolution, does the input octree changes? or it is just for index.
- How can I constract a decoder as the generator of GAN? Should I create a original octree as one of the inputs? And how?
- Is there anything else I need to know for constracting a decoder?
By the way, I am using pytorch.
I build a generator. Is this right?
class G(nn.Module):
def __init__(self, flags):
super(G, self).__init__()
self.depth = 6 # 64**3, voxels
self.deconv_bn_relu_1 = ocnn.OctreeDeConvBnRelu(depth=1, channel_in=200, channel_out=512,
kernel_size=[4], stride=2)
self.deconv_bn_relu_2 = ocnn.OctreeDeConvBnRelu(depth=2, channel_in=512, channel_out=256,
kernel_size=[4], stride=2)
self.predict_2 = self._make_predict_module(channel_in=256)
self.deconv_bn_relu_3 = ocnn.OctreeDeConvBnRelu(depth=3, channel_in=256, channel_out=128,
kernel_size=[4], stride=2)
self.predict_3 = self._make_predict_module(channel_in=128)
self.deconv_bn_relu_4 = ocnn.OctreeDeConvBnRelu(depth=4, channel_in=128, channel_out=64,
kernel_size=[4], stride=2)
self.predict_4 = self._make_predict_module(channel_in=64)
self.deconv_bn_relu_5 = ocnn.OctreeDeConvBnRelu(depth=5, channel_in=64, channel_out=32,
kernel_size=[4], stride=2)
self.predict_5 = self._make_predict_module(channel_in=32)
self.deconv_6 = ocnn.OctreeDeconv(depth=6, channel_in=32, channel_out=1,
kernel_size=[4], stride=2)
self.octree2voxel = ocnn.FullOctree2Voxel(6)
self.sigmoid_6 = torch.nn.Sigmoid()
def forward(self, input):
# input is the latent code Z, with the size of (N, 200, 1, 1, 1)
data = input
# octree = ocnn.octree_new(batch_size=1, channel=4, node_dis=True, adaptive_layer=0)
octree = ocnn.create_full_octree(depth=self.depth, channel=4, batch_size=1, node_dis=True)
data = self.deconv_bn_relu_1(data, octree)
data = self.deconv_bn_relu_2(data, octree)
label = self.predict_2(data)
octree = ocnn.octree_update(octree, label, depth=2, split=1)
data = self.deconv_bn_relu_3(data, octree)
label = self.predict_3(data)
octree = ocnn.octree_update(octree, label, depth=3, split=1)
data = self.deconv_bn_relu_4(data, octree)
label = self.predict_4(data)
octree = ocnn.octree_update(octree, label, depth=4, split=1)
data = self.deconv_bn_relu_5(data, octree)
label = self.predict_5(data)
octree = ocnn.octree_update(octree, label, depth=5, split=1)
data = self.deconv_6(data, octree)
data = self.octree2voxel(data)
data = self.sigmoid_6(data)
return data
@staticmethod
def _make_predict_module(channel_in, channel_out=2, num_hidden=32):
return torch.nn.Sequential(
ocnn.OctreeConv1x1BnRelu(channel_in, num_hidden),
ocnn.OctreeConv1x1(num_hidden, channel_out, use_bias=True))
And I have a few questions about octree code:
Q1: For ocnn.octree_new(), what are the meanings of parameter node_dis, adaptive_layer, and channel? (Is channel the dimension of normals?)
Q2: For ocnn.octree_grow(), what are the meanings of parameter full_octree?
Q3: When to use octree_grow()?
Q4: What are the differences between creating a full octree at the beginning of the forward process and using octree_grow() for each depth during the forward process?
Q5: Does the label in octree_update() decide which nodes in this depth have child nodes?
Q6: What is the function of ocnn.octree_align()? (I mean that if I use ocnn.octree_align(), what would happen?)
Hi, I am trying to build a GAN network with your O-CNN code. Here are some problems I want to ask:
- During the forward process of convolution, does the input octree changes? or it is just for index.
- How can I constract a decoder as the generator of GAN? Should I create a original octree as one of the inputs? And how?
- Is there anything else I need to know for constracting a decoder?
- In the encoder, the octree is fixed. In the decoder, the octree can be predicted dynamically.
- It is non-trivial to build GAN with o-cnn. It is an open question.
- You can refer to the code for shape completion here.
Q1: For ocnn.octree_new(), what are the meanings of parameter node_dis, adaptive_layer, and channel? (Is channel the dimension of normals?)
Q2: For ocnn.octree_grow(), what are the meanings of parameter full_octree?
Q3: When to use octree_grow()?
Q4: What are the differences between creating a full octree at the beginning of the forward process and using octree_grow() for each depth during the forward process?
Q5: Does the label in octree_update() decide which nodes in this depth have child nodes?
Q6: What is the function of ocnn.octree_align()? (I mean that if I use ocnn.octree_align(), what would happen?)
- If
node_dis
= True, we interpret the fourth channel of octree features as the displacement.adaptive_layer
indicates the octree layer where the adaptive octree starts. thechannel
is the channel number of octree features. full_octree
means currently the octree is a full octree- please refer to the code for shape completion for the usage of octree_grow
- There is no big difference.
- Yes.
- The octree_align is the output guided skip connections proposed in our paper for shape completion here
Thanks a lot. O-CNN is a wonderful work. I will keep going on constructing a GAN with the octree.
Regards,
TrepangCat
Hi @TrepangCat ,
I'm curious if you've made progress on your work with constructing a GAN with the octree, and if so, would you be willing to share your implementation?
Sorry, I didn't finish it. There are some problems I can not solve.