libc++abi: terminating with uncaught exception of type vpMatrixException: The algorithm computing SVD failed to converge.
spectra-analytics opened this issue · comments
Hi,
I've been stuck on this for days. I have a working implementation of VISP with AprilTag detection in 2 other apps. I am trying to implement a more "Swift-ier" version (using less obj-c, so the memory might be the issue here) in this app. The error is the title of this issue, and I know it has to be related to the camera intrinsic matrix but everything looks solid to me so I'm finally going to reach out for help.
This is how I am retrieving the intrinsic matrix of the camera:
var matrix = matrix_float3x3.init()
if let camData = CMGetAttachment(sampleBuffer, key: kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, attachmentModeOut: nil) as? Data {
camData.withUnsafeBytes { (buffer: UnsafeRawBufferPointer) in
guard let pointer = buffer.bindMemory(to: matrix_float3x3.self).baseAddress else {
return
}
matrix = pointer.pointee
}
}
let px = matrix.columns.0.x
let py = matrix.columns.1.y
guard let uiImage = imageFromCVPixelBuffer(pixelBuffer: imagePixelBuffer) else { fatalError() }
CVPixelBufferUnlockBaseAddress(imagePixelBuffer,[])
let visp_detector = VISPDetector()
visp_detector.detectTags(uiImage,
cam_px: Double(px),
cam_py: Double(py),
tagSize: Double(0.0315))
The matrix looks like this:
| 1527.41357 0 539.5 |
| 0 1527.41357 959.5 |
| 0 0 1 |
It is a fairly simple implementation as follows:
vpImage<unsigned char> I = [ImageConversion vpImageGrayFromUIImage:frame];
float u0 = I.getWidth()/2;
float v0 = I.getHeight()/2;
if(cam_px == 0.0 && cam_py == 0.0){
cam_px = 1515.0;
cam_py = 1515.0;
}
float quadDecimate = 3.0;
int nThreads = 1;
detector.setAprilTagQuadDecimate(quadDecimate);
detector.setAprilTagNbThreads(nThreads);
vpCameraParameters cam;
cam.initPersProjWithoutDistortion(cam_px,cam_py,u0,v0);
std::vector<vpHomogeneousMatrix> cMo_vec;
detector.detect(I, tagSize, cam, cMo_vec);
Where the parameters look fairly good to me:
cam_px double 1526.1195068359375
cam_py double 1526.1195068359375
u0 float 540
v0 float 960
Small but important update, this only occurs when a tag is in frame. If a tag is not in frame, detect() runs smoothly.
I tried a different purely objective-c++ approach and the algorithm still does not converge.
`- (NSArray *)detectTags:(CMSampleBufferRef)frame
cam_px:(double)cam_px
cam_py:(double)cam_py
tagSize:(double)tagSize
{
CVImageBufferRef buffer = CMSampleBufferGetImageBuffer(frame);
CVPixelBufferLockBaseAddress(buffer, 0);
void *baseAddressY = CVPixelBufferGetBaseAddressOfPlane(buffer, 0);
int bufferWidthY = (int)CVPixelBufferGetWidthOfPlane(buffer, 0);
int bufferHeightY = (int)CVPixelBufferGetHeightOfPlane(buffer, 0);
int bytePerRowY = (int)CVPixelBufferGetBytesPerRowOfPlane(buffer, 0);
/* create grayMat using obtained parameters */
cv::Mat grayMat = cv::Mat(bufferHeightY, bufferWidthY, CV_8UC1, baseAddressY, bytePerRowY);
vpImage<unsigned char> I;
vpImageConvert::convert(grayMat, I);
float u0 = I.getWidth()/2;
float v0 = I.getHeight()/2;
if(cam_px == 0.0 && cam_py == 0.0){
cam_px = 1515.0;
cam_py = 1515.0;
}
float quadDecimate = 3.0;
int nThreads = 1;
detector.setAprilTagQuadDecimate(quadDecimate);
detector.setAprilTagNbThreads(nThreads);
vpCameraParameters cam;
cam.initPersProjWithoutDistortion(cam_px,cam_py,u0,v0);
std::vector<vpHomogeneousMatrix> cMo_vec;
detector.detect(I, tagSize, cam, cMo_vec);
int tagNums = (int) detector.getNbObjects();
for(int i=0; i < tagNums; i++){
std::vector<vpImagePoint> polygon = detector.getPolygon(i);
vpImagePoint cog = detector.getCog(i);
vpTranslationVector trans = cMo_vec[i].getTranslationVector();
}
CVPixelBufferUnlockBaseAddress(buffer, 0);
return nullptr;
}`
Difficult to figure out what is wrong!
Some suggestions:
- The error occurs in
svdLapack()
that is called during pose estimation. You may try to see if the tag detection is working without estimating the pose, just to confirm that the pose algorithm is throwing the exception. To do that call simplydetector.detect(I, tagSize, cam);
and do not usecMo_vec
- If the previous test is working without throwing an exception, try to change the pose estimation algorithm using setAprilTagPoseEstimationMethod() or directly in the
vpDetectorAprilTag
constructor - If Eigen or OpenCV 3rd parties available, disable Lapack usage
cmake ../visp -DUSE_BLAS/LAPACK=OFF -DUSE_EIGEN3=ON; make
and try again (similar with OpenCV)
bool detect(const vpImage<unsigned char> &I); bool detect(const vpImage<unsigned char> &I, double tagSize, const vpCameraParameters &cam, std::vector<vpHomogeneousMatrix> &cMo_vec, std::vector<vpHomogeneousMatrix> *cMo_vec2=NULL, std::vector<double> *projErrors=NULL, std::vector<double> *projErrors2=NULL);
cMo_vec is not an optional parameter, so I'm not sure how I could execute that test. Correct me if wrong, on VISP 3.5.0
Either way, the situation has magically resolved itself by reimporting VISP. I did reimport it several times over the past few days, but finally decided to just redownload and reimport and that one worked. Very odd behavior, something had to be corrupted.