mochsner / STEREO-RECONSTRUCTION

STEREO-RECONSTRUCTION

Stereo reconstruction is a method by which we obtain the depth in a particular scene by using a pair of stereo image.

what are stereo images:

pair images of same scene which are taken from a different angle or perspective

Algorithm explanation:

disparity and disparity maps:

disparity is defined as the difference of position of a sepcific point or object in right image to left image.

Z=B*f/D

where, f is focal length

B is baseline length

Z is depth

D is disparity

using epipolar geomerty we obtain depth by calculating disparity

principle :

epipolar geomerty:

epipolar geomerty gives us a geometric relation between pair of setreo images from which depth of a point can be calculated. This geometric relation is known as epipolar constraint according to it a point in first images lies on the epipolar line of second image.

epipolar line is line of intersection of image plane and palne containing optical centres of stereo cameras and point in real world.

epipolar geomerty basically converts 2D search into 1D search for stereo correspondance.

code:

1. first install opencv and matplotlib library

pip install matplotlib

pip install opencv-python

2. load images in grayscale using cv.imread

• parameters of cv.imread:

  • image path
  • image colour

3. after loading images , use cv.StereoSGBM_create or cv.StereoSGBM_create .

• paramters of cv.StereoSGBM_create:

  • mindisparity

  • numDisparities = (maxdisparity - mindisparity):this parameter much divisble by 16

  • blockSize(should be a odd number)

  • P1 & P2(controls smoothness of disparity map)

  • uniqueness Ratio

  • speckleWindowSize

  • speckleRange

    click here for more parameters

4. compute disparity using stereo.compute

• parameters of stereo.compute :

  • right image
  • left image

5. obtained disparity .
6. apply gaussian blur on disparity for better results
7. for visualization of disparity map we use matplotlib function plt.imshow

• parameters of plt.imshow

  • disparity or any other image we want to show.

  • colormap(for current project we use jet colormap)

    for more info about colormaps click here.

8. then pass plt.show to display output.

reprojection of image points(u,v) to 3D coordinates:

principle:

to convert 3D points(X,Y,Z) into 2D image coordinates(U,V) for this we use internsic camera matrix.

here Fx and Fy are focal length

S skew of camera

X0 and Y0 are camera offsets we use calibartion matrix to obtain 3D coordinates using disparity.

code :

9. after visualization , use cv.reprojectImageTo3D to obtain 3D coordinates of points.

• parameters of cv.reprojectImageTo3D :

  • diparity map
  • calibration matrix (numpy array containing camera parameters)

10. obtain point's colour using cv.cvtColor

• parameters of cv.cvtColor :

  • image
  • color space

11. thresholding 3D points to remove points having no diparity or very large depth.

creating point cloud:

code:

12. create point cloud using ply file

13. define elements and property of elemenst using ply header

14. save all the points and their respective in colours in .ply file

15. finally open .ply file in meshlab to visualize the point cloud

output :

image 1:

image 2:

disparity map:

result :

To run code :

git clone https://github.com/yaswanth1701/STEREO-RECONSTRUCTION.git

• edit image path and run.