3D reconstruction from 2D images pipeline
Steps:
- Detect 2D points
- Match 2D points across 2 images
- Epipolar geometry
3a. If both intrinsic and extrinsic camera parameters are known, reconstruct with projection matrices.
3b. If only the intrinsic parameters are known, normalize coordinates and calculate the essential matrix.
3c. If neither intrinsic nor extrinsic parameters are known, calculate the fundamental matrix. - With fundamental or essential matrix, assume P1 = [I 0] and calulate parameters of camera 2.
- Triangulate knowing that x1 = P1 * X and x2 = P2 * X.
- Bundle adjustment to minimize reprojection errors and refine the 3D coordinates.
Note: Steps and code in this repo is my hobby / learning exercise. Ie, its probably not very efficient. If you wish to use a more production-ready library, check out OpenCV's SFM module. I have a docker environment for it at: https://github.com/alyssaq/reconstruction
- Python 3.5+
- Install OpenCV: Mac installation steps
- pip install -r requirements.txt
- python3 -m pip install opencv-python==3.4.2.16
- python3 -m pip install opencv_contrib_python==3.4.2.16
$ python3 cube_reconstruction.py
Download images from http://www.robots.ox.ac.uk/~vgg/data/data-mview.html and place into imgs/dinos
$ python3 example.py
Detected points and matched across 2 images.
3D reconstructed dino with essential matrix
$ python3 camera.py
3D points of model house from Oxford University VGG datasets.
- Oxford University, Visual Geometry Group: http://www.robots.ox.ac.uk/~vgg/data/data-mview.html
- EPFL computer vision lab: http://cvlabwww.epfl.ch/data/multiview/knownInternalsMVS.html