Question about image hessian
HCSong007 opened this issue · comments
Hi there,
Thank you for the wonderful job. In your codes, subview image hessian is used to calculate the jacobian entries, which is not mentioned in your paper. Could you please explain it a little bit. Appreciate it.
Sure!
The geometric loss function compares image gradients. To build the Jacobian for the gauss newton iterations we need to differentiate this loss function with respect to the depth that we want to optimize for.
If we write the loss in a slightly abstract way we would have loss = image_gradient(projection(depth)), i.e. the loss depends on the image gradient (in a neighbor view) at at location that depends on the projection according to the currently estimated depth.
Differentiating this wrt. the depth means we have to apply the chain rule. So for the gradient of the loss function we get a product of the image gradient itself differentiated wrt. the image coordinates as the output of the projection function, times the projection function differentiated wrt. the depth.
The image gradient differentiated wrt. the image coordinate is the Hessian that is used in the code.
Does that help?
It helps a lot. Thank you to make it clear to me. Appreciate it.
So, based on my understanding, if we do not take shading term into consideration right now, the geometric loss can only provide update for depth, and for the rest three coefficients, like dx, dy & dxy, their updates all rely on the smoothness term which is the divergence of normal, am i right?
Technically the geometric loss also puts constraints on all coefficients of the patches. In the end the depth itself is another function of the patch coefficients, so this would be another application of the chain rule. However, the geometric loss alone is not very stable wrt. dx, dy, and dxy, so we need another term (either regularization or shading) to make the optimization more stable for these coefficients.