TL; DR: Jump to Installation and Tutorial.
This repository is slightly modified from the original multi-view camera calibration repository here to make it work in Linux environment and a more casual setting (7 RealSense 435 cameras, no depth used, highly different viewing positions and angles). The original code was based on Windows and tested with 16 high performing RGB cameras that are closely adjacent with similar viewing angles.
I (Hongsuk Choi) only modified a small portion of
- Installation guidelines for Linux environment
- C++ code of Ceres bundle adjustment
- Initial multi-view calibration before bundle adjustment
Great thanks to Hyojoon Park for the awesome code. Contact to redstonepo@gmail.com if anyone has questions about the code I modified.
This project calibrates multiple cameras using a planar calibration checkerboard. The pipeline is comprised of four main steps: (1) detect checkerboard corners, (2) remove outlier corners from the previous step (using VAE, variational auto-encoder), (3) estimate initial camera parameters and world (checkerboard) points, (4) refine the initial estimates (using bundle adjustment), then optionally (5) analyze the calibration result.
- This calibration method has been used for "Capturing Detailed Deformations of Moving Human Bodies". Chen He, Hyojoon Park, Kutay Macit, and Ladislav Kavan. SIGGRAPH 2021.
- This project is focused on being simple and scalable, applicable to different calibration checkerboards and different number of cameras.
- The steps (1)-(3)/(5) are implemented using python 3 and the step (4) using C/C++.
- On the example dataset provided (./example_data/), this project achieves mean reprojection error 0.08 pixels with standard deviation 0.05.
The code execution follows the pipeline below:
(Input) (0). Multi-view images: The input is a set of images capturing a freely moving checkerboard.
(1). Corner detection: Checkerboard corners are detected and localized with sub-pixel precision using OpenCV.
(2). VAE outlier detector: Outlier corners are identified using VAE (variational auto-encoder), and corresponding images are discarded. As expained here (Section 3), open-source corner detectors (e.g., OpenCV, MATLAB, etc.) are known to give incorrect or inaccurate (i.e., outlier) corners which may deteriorate the accuracy of camera caibration. Therefore, we identify and remove such outliers by tranining a VAE outlier detector. Its effectiveness is explored more here (Section 4).
(3). Initial camera calibration: Initial camera parameters and frame-wise checkerboard poses are first estimated from a subset of images using OpenCV.
(4). Bundle adjustment: The estimated camera parameters and frame-wise checkerboard poses are further refined using bundle adjustment.
(Optional) (5). Analyze the calibration result: Computes reprojection errors and render corresponding histogram, reprojected images, etc. for analysis.
(1) Download the project. In a command prompt (Windows) or terminal (Linux):
git clone https://github.com/hjoonpark/MultiCamCalib.git
(2) Install Ceres
(3) Build the bundle adjustment project (C/C++)
- Click here
(4) Create Anaconda environment for Python
-
Windows
Assuming Anaconda is already installed,
conda env create --file windows_env.yml -
Linux
Install appropriate Python and Torch versions, in case you want to build an api from source.
conda env create -n multicamcalib python=3.8 pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113Install requirements
pip install -r requirements.txt
(5) Run the codes → Follow this tutorial!
Follow this quick start tutorial (a step-by-step example). After you calibration your own data, it is highly recommended to validate the result by stitching point clouds, if you have depth images. Refer to the ./multicamcalib/valdiate.py
Click here
For questions, please contact Hyojoon Park: hjoonpark.us@gmail.com