catree / tf_calib

Tensorflow camera calibrator

Tensorflow calibrator

Tensorflow camera calibrator

Prerequisites

sudo pip3 install numpy, scipy, tensorflow_gpu

The example data is collected using my Logitech C 310 webcam.

After 10 epochs we reach precision < 10^-2 size of the quare (~7cm)

Theory

Pinhole camera

Our goal is to find the camera matrix K together with distortion coefficients, the so-called intrinsic parameters of the camera.

The camera matrix reads:

,

where f_x, f_y are focal lengths of the camera and c_x, c_y are the pixel coordinates of the principal point.

The distortion coefficients depend on the model. Currently we have implemented only simple radial distortion coefficients k₁, k₂, k₃ from OpenCV docs.

The world coordinates (x₁,x₂,x₃) in the camera coordinate system are related to the pixel coordinates (ξ₁,ξ₂,1) via the camera matrix as follows:

with a scale parameter s, which corresponds to the arbitrary scale due to the fact that the camera projects an entire ray to a single point.

The inverse of this relation looks as follows:

(1)

The vectors uⁱ define the ray corresponding to the i-th point.

Minimization

Let cⁱ_α be the coordinates of a calibration pattern in its own coordinate system. Here and further the latin index i runs from 1 to N, where N is the number of points in the calibration pattern. Greek indices α,β... run from 1 to 3. Note that we do not assume anything about the structure of the calibration pattern.

Now, let ξⁱ_α be the pixel coordinates of the ith pattern point on the image.

Assuming the camera intrinsic parameters are known we can find a rotation matrix R and a translation vector T which, together with the scale parameters sⁱ, minimize the following error:

The quantities u are defined in (1). We equipped the variables from the previous section with additional index i, which indicates the number of the point in the calibration pattern.

Indeed, sequientially taking the derrivatives of E by sⁱ, T, and R and equaling them to zero we obtain the following expressions for (1):

(2)

Here we defined the following auxilliary quantities:

The projector alogn uⁱ

,

The projection of the rotated vector Rcⁱ on the line uⁱ

,

The sum of projectors and its inverse:

,

With this notations the matrix U looks:

, (3)

where J ^N is NxN all-one matrix matrix and I 's are identity matrices of the corresponding dimensions.

The definition (3) in the operator notations reads:

The rotation matrix is determined from the following equation:

(4)

where the matrix M reads:

To summarize, the algorithm for determining the parameters K is the following:

1. Initialize K
2. For each image
   1. Solve the equation (4) with respectto R subject to R^TR = I
   2. Apply gradients descent to parameters K with loss function (2)

Note that the parameters K are hidden in the vectors u.