How to set max depth with usb streaming, or why is max depth 1m?
Michelvl92 opened this issue · comments
I am using the Record3D app for streaming over USB to your Record3D Python API.
The problem is that when the RGBD data is streamed over USB, the maximum depth map shown in the demo-main.py app looks if it is not more than 1 meter. Everything more than 1 meter is shown white. So, by default what is the maximum depth that can be recorded (over Wi-Fi it says 3 meters)? or is there a possibility for usb-recoding to get recording of more than 1 meter?
Furthermore, when recording with option “higher-quality LiDAR recording” the depth map has a resolution of 192x256 with the iPad Pro (12.9 inch) 4th gen (due to its sensor limitation). Is it True that the iPhone 12/13/14 pro have LiDAR sensor 640 x 480, or 0.3MP, and is it possible to stream this resolution over USB?
Ipad specs:
• iPad Pro (12.9 inch) 4th gen
• iPadOS version: 16.5
• Record3D version: 1.8.6
Record3D the accompanying library: 1.3.1.
There is no limitation on maximum depth during USB streaming. Try normalizing the depth maps and you should see that the camera captures scene beyond 1 meter.
The reason why you see white color is just a side effect of the chosen visualization; the depth maps are stored as float32 arrays, but OpenCV converts them into uint8 — likely by multiplying the float32 values it by 255, and since uint8 cannot hold a value greater than 255, you see the >1 m pixels as white because OpenCV clamps values greater than 255 to 255.
Is it True that the iPhone 12/13/14 pro have LiDAR sensor 640 x 480, or 0.3MP, and is it possible to stream this resolution over USB?
No, that is not true. LiDAR resolution is still just 192x256 px on all iPhones and iPads so far.
Thanks, that makes everything very clear.
Do you also have suggestions to reduce and/or remove blurry frames?
Furthermore, is there also an option to get the timestamp when getting a frame? Furthermore, is it also possible to the fps from the device?
Do you also have suggestions to reduce and/or remove blurry frames?
Unfortunately, there is no way to do this.
Furthermore, is there also an option to get the timestamp when getting a frame?
This is not implemented. I think that getting the true UNIX timestamp of when a camera frame has been captured is not currently possible with Apple's APIs (if anyone knows how to do this, please let me know).
Furthermore, is it also possible to the fps from the device?
If you meant if it is possible to query or change the camera FPS using this library, then that is not possible. If you wanted to ask whether is it possible to somehow change the camera FPS, then yes — you can manually change FPS by going to the Settings tab > Animation Rate (FPS) > [select the FPS you want].