Using Hand Tracker with pre-recorded sequences
olkido opened this issue · comments
Hi,
I would like to use the Hand Tracker (for now just the SingleHandTracking.py) with some RealSense sequences that I have previously recorded and saved to image files. Ideally I would like to simply load opencv images, i.e. not go through an .oni file as suggested here #42 . Is this possible? Could you perhaps help me out with some tips on how to set the calibration data (variable clbs if I understand correctly) (formats etc) and the image data (variable imgs)?
Thanks very much.
Image acquisition is performed at line 122. You can replace the default acquisition with any code that can produce a pair of images (RGB and depth) and some calibration information.
Consider replacing with the following:
import cv2 as cv
# It is assumed that both RGB and depth streams have the same dimensions as images
# The calibration used corresponds to the depth stream
imgs = [ cv.imread(pathToDepthImage, cv.IMREAD_UNCHANGED), # Make sure 16bit loading is perserved
cv.imread(pathToRGBImage) ]
cfr = Core.CameraFrustum()
cfr.setIntrinsics(fx, fy, cx, cy, width, height, zNear, zFar) # assuming some focal lengths fx, fy and principal point cx, cy, image size width X height and a reasonable clipping plane for near (zNear) and far (zFar) in the target units (e.g. millimeters).
cfr.setExtrinsics(translation, rotation) # assuming a 3D translation and rodrigues rotation. Optional.
clb = Core.CameraMeta(cfr, width, height) # assuming the width and height of the images in pixels
Thanks @nkyriazis ! I'm getting the following error
File "src/SingleHandTracking.py", line 143, in
cfr.OpenCV_setIntrinsics(fx, fy, cx, cy) # assuming some focal lengths fx, fy and principal point cx, cy.
Boost.Python.ArgumentError: Python argument types in
CameraFrustum.OpenCV_setIntrinsics(CameraFrustum, float, float, float, float)
did not match C++ signature:
OpenCV_setIntrinsics(MBV::Core::CameraFrustum {lvalue}, cv::Mat, cv::Size_, float, float)
By doing nm ../MBV_PythonAPI_Linux_1.1/python_libs/PyMBVCore.so | c++filt | grep -i setintrinsics
I get the following signatures:
U MBV::Core::CameraFrustum::setIntrinsics(float, float, float, float, float, float, float, float)
U MBV::Core::CameraFrustum::OpenCV_setIntrinsics(cv::Mat const&, cv::Size_ const&, float, float)
It seems like I need extra parameters? I simply gave numerical values to fx ,fy, cx, cy, should they be something else?
Thanks again.
My bad. I updated the code to better reflect the API. If the nature of the extra parameters is still not clear I can come back with more details.