Geometry consistency check won't pass if two images are exactly the same
symao opened this issue · comments
I found that when i push the same image into loopdetect twice, two images' keypoints, descriptors and feature vectors are exactly the same, the DBoW2 gives the score 1, but the geometry consistency check is not passed. But if two image is not the same, but very similar, for example DBoW2 score is less than 1, geometry consistency check will pass. This is counterintuitive, cause if two image are the same, means they are infinitely similar to each other, loop detect should return the 'true' flag.
I was use isGeometricallyConsistent_DI() and direct_level is 2.
Although we won't capture two image exactly the same, even in the same spot. But i still want to solve this extreme case. Now, i just skip the geometry consistency if dbow2 score are very large. Is there a better way to solve this?
The geometry consistency check is quite conservative now. It tries to fit a fundamental matrix between the two images, but this isn't well-conditioned in some cases, such as when there is no motion between the two images (two images are exactly the same), when the motion is a pure rotation, or when all the corresponding points lie in a planar surface. In those cases, fitting a homography instead of a fundamental matrix would make it.
The solution I see to allow your scenario is to add a homography check to the geometry check. You can do that by adding a line in TemplatedLoopDetector.h, lines 1080, 1140, 1244. You need to instantiate a DVision::HSolver first, as in line 523. Just search for m_fsolver in that class and replicate every occurrence with the HSolver object.
Another option you may try is to comment out checking the condition number of the matrix that generates the fundamental matrix in DVision/FSolver.cpp line 349. This would pass all those cases that are not well conditioned. However, in those cases, there are infinite fundamental matrices that fit the corresponding points, so you might obtain a wrong fundamental matrix that creates very large reprojection errors, preventing the geometry check from passing.
@dorian3d Thank you for your answer. I didn't consider the bad fundamental matrix problem before. The HSolver solution might not suit me, because i want to estimate motion between two images later.
I think that make sense that return 'false detect' if there is no motion or pure rotation, since we cannot calculate the right transformation from fundamental matrix in these cases.
However i still want to do this: If the DBoW2 score get 1, return 'true' detection flag and assume that there is no motion between two images. Do you think that is OK? Although that is ugly :(
In that case, you can just check if island.best_score ~ 1 to set detection = true before the geometry check, in TemplatedLoopDetector.h, line 777: