Optimization

[Andrea Pennisi]

Hi,

I was testing your code and I noticed 2 things:

1 - in the function refine_homographies(H, correspondences, skip=False)
you pass the correspondences and then inside the code you use correspondence, where is it defined??

2- No optimization has been performed by the code. I was taking a look at the jacobian computation, and it seems good.

Can you help me?
Thanks

[Andrea Pennisi]

I found the errors, you pass the old homography as initial_guess and as parameter, so, you modify the initial guess variable but inside the jacobian as well as in the minimizer function, you use always the variable h that does not change.
So, the new functions are:

def minimizer_func(h, X, Y, N):
# X : normalized object points flattened
# Y : normalized image points flattened
# h : homography flattened
# N : number of points
#
x_j = X.reshape(N, 2)
#Y = Y.reshape(N, 2)
# h = h.reshape(3, 3)
projected = [0 for i in range(2*N)]
for j in range(N):
x, y = x_j[j]
w = h[6]x + h[7]y + h[8]
# pts = np.matmul(np.array([ [h[0], h[1], h[2]] , [h[3], h[4], h[5]]]), np.array([ [x] , [y] , [1.]]))
# pts = pts/float(w)
# u, v = pts[0][0], pts[1][0]
projected[2j] = (h[0] * x + h[1] * y + h[2]) / w
projected[2j + 1] = (h[3] * x + h[4] * y + h[5]) / w

return (np.abs(projected - Y))**2

def jac_function(h, X, Y, N):
x_j = X.reshape(N, 2)
jacobian = np.zeros( (2*N, 9) , np.float64)
for j in range(N):
x, y = x_j[j]
sx = np.float64(h[0]x + h[1]y + h[2])
sy = np.float64(h[3]x + h[4]y + h[5])
w = np.float64(h[6]x + h[7]y + h[8])
jacobian[2j] = np.array([x/w, y/w, 1/w, 0, 0, 0, -sxx/w**2, -sxy/w2, -sx/w2])
jacobian[2j + 1] = np.array([0, 0, 0, x/w, y/w, 1/w, -syx/w**2, -syy/w2, -sy/w2])

return jacobian

def refine_homographies(H, correspondences, skip=False):
if skip:
return H

image_points = correspondences[0]
object_points = correspondences[1]
normalized_image_points = correspondences[2]
normalized_object_points = correspondences[3]
N_u = correspondence[4]
N_x = correspondence[5]
N_u_inv = correspondence[6]
N_x_inv = correspondence[7]

N = normalized_object_points.shape[0]
X = object_points.flatten()
Y = image_points.flatten()
h = H.flatten()
h_prime = opt.least_squares(fun=minimizer_func, x0=h, jac=jac_function, method="lm" , args=[X, Y, N], verbose=0)


if h_prime.success:
    H =  h_prime.x.reshape(3, 3)
H = H/H[2, 2]
return H