HybVIO assume/tuned for parallel stereo cameras?
ArmandB opened this issue · comments
I've been using HybVIO on the recently released Monado VR dataset: https://huggingface.co/datasets/collabora/monado-slam-datasets and have been observing a lot of failures, even after increasing the parameters like BA problem size and pose history size in order to account for the higher fps (54).
In many cases, the number of tracks is extremely low (many new key points are detected in the left frame but very few are tracked to the right frame), but tracking proceeds much better when I enable the "useRectification" flag. When I contacted the dataset's author, they said that when they tuned Basalt to run on it, they had to modify the algorithm because it was tuned for parallel stereo cameras and the Valve Index (the VR headset the data is from) cameras are canted.
Just wanted to ask and see if this might be what's causing the problem here as well.
Thank you in advance!
Hi Armand. The HybVIO codebase would need numerous changes to work for real-time inside-out tracking in an actual headset. Supporting non-parallel stereo cameras (which is currently not properly supported, like you suspected) is just one of them.
These features are included in our commercial SDK, which we recommend for all real-time embedded use cases.
Also already discussed in #22
Dear Otto,
Sorry for the duplicate question and thank you so much for the speedy response!
I had looked at #22 about a year ago and incorrectly remembered that it was only about HybVIO not supporting key points in the non-overlapped camera FoV.
Thank you again for your help!