Weired fine-detailed reconstruction for can class
HanzhiC opened this issue · comments
Hey I was testing the reconstruction quality of the byproduct reconstruction. The meshes of the other objects look good to me except for the Can class, which always gives out tiny shapes.
I think the pose and scale are correctly applied to transform mesh from canonical space to camera space, as the other objects look very reasonable. I suspect there are some problems with the pre-trained DualSDF-weight, as I observe that during shape optimization in the optimize_shape_ransac function, the decoded shape from the feature is always tiny.
could you kindly see what is going on with the mesh reconstruction?
Here are some examples:
Thank you!
Hi, actually we met this problem before.
We think this is because the number of CAD models (can category) used to train DualSDF is small. This makes the fine shape not consistent with the optimized coarse shape.
To show the mesh reconstruction in the supplementary material, we made data augmentation on given CAD models (like stretching the cans) to make the DualSDF more stable. To evaluate the reconstructed shapes, you could use the coarse shapes, or try the above method to make DualSDF stable.
Hi, actually we met this problem before.
We think this is because the number of CAD models (can category) used to train DualSDF is small. This makes the fine shape not consistent with the optimized coarse shape.
To show the mesh reconstruction in the supplementary material, we made data augmentation on given CAD models (like stretching the cans) to make the DualSDF more stable. To evaluate the reconstructed shapes, you could use the coarse shapes, or try the above method to make DualSDF stable.
Could you elaborate on what you mean by "using coarse shapes" (low resolution for marching cube, or use prim_sdf_fun as rendering function, or?)? Also, is this data augmentation used in test or train time?
Could you elaborate on what you mean by "using coarse shapes" (low resolution for marching cube, or use prim_sdf_fun as rendering function, or?)? Also, is this data augmentation used in test or train time?
Coarse shape means semantic primitives, the spheres generated by prim_sdf_fun. You could treat the spheres' centers as the reconstructed point cloud, and measure the chamfer distance between the reconstructed point cloud and the ground truth shape as shown in the paper.
The above data augmentation is used to expand the training samples of CAD models to pre-train a DualSDF model of the can category.
Thanks for such a quick response. Now see the point.
One last question, is the weight pre-trained with such a data augmentation strategy available?
Yes, but cannot provide in time due to lab closure during spring festival.
Take your time. It would be wonderful if you could provide it some time.
Thank you.
Hi, you could try to replace the weights of can category with epoch_9999.pth and replace the segmetation weights with epoch_14_iters_290640.pth.