about dL_dmean2D
CanCanZeng opened this issue · comments
Hi, thank you for sharing this reproduce work. Your notes are very helpful for 2DGS learners.
When I checked the code, I found that it seemed that dL_dmean2D was not calculated. If dL_mean2D was not implemented, how would the algorithm work? What I mean is that the later stage of densification depends on the gradient of the projected point.
I'm trying to implement AbsGS based on 2DGS, but since the projection points of 2DGS are different from 3DGS, obtaining the Abs gradient of the projected point is not trivial. Can you give me some help?
Thank you for your reply.
I think I made a mistake yesterday. I only noticed the variable dL_dmean2D and didn't notice that the gradient of the image point you returned is p2_gradient.
Can I assume that only when dist3 is greater than dist2 does the loss have a gradient on the image point? Otherwise, the other case that the computational flow follows is independent of the image point.
If my idea is correct, then this gradient does not seem to align with the intention of the 3DGS authors. The 3DGS authors observed that the over and under reconstruction areas had a greater gradient on the projected point, so they chose this indicator as the criterion for whether to clone or split the Gaussian.
Can I assume that only when dist3 is greater than dist2 does the loss have a gradient on the image point? Otherwise, the other case that the computational flow follows is independent of the image point. If my idea is correct, then this gradient does not seem to align with the intention of the 3DGS authors. The 3DGS authors observed that the over and under reconstruction areas had a greater gradient on the projected point, so they chose this indicator as the criterion for whether to clone or split the Gaussian.
You are correct in your understanding of the gradients for dist3 and dist2. However, the concept behind 2DGS is not to use the gradients of projection points on the image to represent whether the scene is under-reconstructed or over-reconstructed. Instead, it involves projecting the gradients of three-dimensional points onto the image. If the scene is either over-reconstructed or under-reconstructed, there will similarly be significant gradients of the three-dimensional points, which is consistent with 3DGS.
The reason 2DGS operates this way is that the projection transformation is not affine. Projecting the Gaussian centroids onto the image might not align with the center points of the two-dimensional Gaussian patterns. Therefore, they believe that projecting the gradients of the three-dimensional centroids onto the image plane more accurately reflects the "severity of the actual Gaussian movement." You can visualize this in the official code’s Python demo by plotting the projection points of the Gaussian centroids and the center points of the bounding boxes; you will notice a slight difference between the two.