Termination at Non-overlapping Calibration

[Benjamin Heuberger]

System information (version)

• Operating System / Platform => Docker Image on Ubuntu 22

Vision system

• Number of cameras => 2
• Types of cameras => perspective (perspective, fisheye, hybrid)
• Multicamera configurations => overlapping (overlapping, non-overlapping, converging)
• Configuration file => 1_calib_param.yml (i.e. *.yml)
• Image sequences =>
  • 26 images per camera
  • link to google drive can be found below

Describe the issue / bug

First thanks a lot for your work. I tried to calibrate two overlapping cameras using the docker image. The calibration process seems to work till it comes to the point where it tries to calibrate Non-overlapping cameras (which there are none in this setup). Then it aborts with the following error:
terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.2.0) ../modules/core/src/kmeans.cpp:242: error: (-215:Assertion failed) data0.dims <= 2 && type == CV_32F && K > 0 in function 'kmeans'
I also tried to install MC-Calib without the Docker image but the error still persists.
The whole output can be found here:
calibrate_output.txt
The calibration images can be found here:
Calibration Images on Google Drive

[rameau-fr]

Dear Benjamin,

Thank you very much for using MC-Calib. I have checked the images you shared in detail, and your use case is very interesting.

I identified exactly what your problem was; even if the cameras share an overlapping field of view, your checkerboard is too small and is never detected simultaneously by both cameras. Therefore, MC-Calib is trying to calibrate these two cameras as if they were non-overlapping but failed because it is, in this case, impossible.

I've verified it by calibrating each camera individually. Camera 1 detects the boards at the image indexes: 3,4,5,6,7,8,9,10,22,23. Camera 2 had successful detections at indexes: 2,12,14,15,16,18,24,25,26.

Considering the difficulty of your task, I'd like to recommend some alternative solutions:

Calibration in Two Stages (More Precisely, Three Stages) with Two Boards:

1. First, calibrate only the intrinsic properties for both cameras individually. Ensure the board covers the entire image and try to collect many images.
2. Print a larger checkerboard or a similarly sized one with fewer squares to enhance detection. Aim to capture many overlapping images.
3. Run the calibration with fixed intrinsic parameters to estimate only the extrinsic params.

Single Sequence Calibration Using Two Checkerboards:

Alternatively, instead of multiple calibration stages, print two checkerboards: a large one for extrinsic calibration and a smaller one for intrinsic. Start by moving the smaller board in front of camera 1 (and then the same for camera 2). After, you hide the small board and focus on extrinsic calibration using the larger board. Finish by running MC-Calib for the entire sequence. This method should provide good results for both intrinsic and extrinsic calibration.

P.S: If you can acquire a larger checkerboard with bigger corners, it might be possible to calibrate the entire system in a single sequence. But, you'd likely need more than 26 images for best accuracy.

Additional Notes:

• For single camera calibration: Since CCTVs often use CMOS sensors with noticeable rolling shutter effects, I recommend a tripod and a holder for the calibration board.
• I noticed some images with motion blur. It might help to stabilize the checkerboard, adjust camera exposure, or improve room lighting.
• For multiple cameras: Synchronization is vital. If you lack a precise electronic shutter control system, achieving proper synchronization might be a challenge. Thus, hold the board during captures.
• For larger checkerboards: Size does matter, but if you print a larger one you need to make sure the surface is perfectly planar and that it does not bend. So it is not easy to have such properties for large surfaces.

What goals do you have for your system? What level of accuracy are you aiming for?

In short:

1. Consider a larger checkerboard or one with fewer squares.
2. Ensure your cameras are well-synchronized and global shutter. If not, keep the board static during captures.
3. If calibration issues arise, think about multi-stage calibration.
4. If none of this works and top accuracy isn't essential, I might have further suggestions ;-).

Best,

Francois

[Benjamin Heuberger]

Dear Francois

Thank you very much for the fast and detailed answer! I am going to try your solutions in the next couple of days and will come back to you after that.

The system does not have to be 100% accurate, as it is just used to estimate where people are standing in the 3D space within the region of the cameras. I just need to limit the distortions at the borders of the image and I need to know, how the cameras are positioned relative to each other. But ultimately, if the extrinsic calibration does not work, I can still use a simple tape measure :P

Best,

Benjamin

[Benjamin Heuberger]

Dear Francois

I now tried it with a bigger board (double the size of the one in the pictures) and with fewer squares (4x3), and it worked like a charm. Thank you very much for your support and software!

Best,

Benjamin