Using Hand Tracker with pre-recorded sequences

[Olga Diamanti]

Hi,
I would like to use the Hand Tracker (for now just the SingleHandTracking.py) with some RealSense sequences that I have previously recorded and saved to image files. Ideally I would like to simply load opencv images, i.e. not go through an .oni file as suggested here #42 . Is this possible? Could you perhaps help me out with some tips on how to set the calibration data (variable clbs if I understand correctly) (formats etc) and the image data (variable imgs)?
Thanks very much.

[Nikolaos Kyriazis]

Image acquisition is performed at line 122. You can replace the default acquisition with any code that can produce a pair of images (RGB and depth) and some calibration information.

Consider replacing with the following:

import cv2 as cv

# It is assumed that both RGB and depth streams have the same dimensions as images
# The calibration used corresponds to the depth stream
imgs = [ cv.imread(pathToDepthImage, cv.IMREAD_UNCHANGED), # Make sure 16bit loading is perserved
         cv.imread(pathToRGBImage) ]
cfr = Core.CameraFrustum()
cfr.setIntrinsics(fx, fy, cx, cy, width, height, zNear, zFar) # assuming some focal lengths fx, fy and principal point cx, cy, image size width X height and a reasonable clipping plane for near (zNear) and far (zFar) in the target units (e.g. millimeters).
cfr.setExtrinsics(translation, rotation) # assuming a 3D translation and rodrigues rotation. Optional.
clb = Core.CameraMeta(cfr, width, height) # assuming the width and height of the images in pixels

[Olga Diamanti]

Thanks @nkyriazis ! I'm getting the following error

File "src/SingleHandTracking.py", line 143, in
cfr.OpenCV_setIntrinsics(fx, fy, cx, cy) # assuming some focal lengths fx, fy and principal point cx, cy.
Boost.Python.ArgumentError: Python argument types in
CameraFrustum.OpenCV_setIntrinsics(CameraFrustum, float, float, float, float)
did not match C++ signature:
OpenCV_setIntrinsics(MBV::Core::CameraFrustum {lvalue}, cv::Mat, cv::Size_, float, float)

By doing nm ../MBV_PythonAPI_Linux_1.1/python_libs/PyMBVCore.so | c++filt | grep -i setintrinsics
I get the following signatures:
U MBV::Core::CameraFrustum::setIntrinsics(float, float, float, float, float, float, float, float)
U MBV::Core::CameraFrustum::OpenCV_setIntrinsics(cv::Mat const&, cv::Size_ const&, float, float)

It seems like I need extra parameters? I simply gave numerical values to fx ,fy, cx, cy, should they be something else?

Thanks again.

[Nikolaos Kyriazis]

My bad. I updated the code to better reflect the API. If the nature of the extra parameters is still not clear I can come back with more details.