libc++abi: terminating with uncaught exception of type vpMatrixException: The algorithm computing SVD failed to converge.

[spectra-analytics]

Hi,

I've been stuck on this for days. I have a working implementation of VISP with AprilTag detection in 2 other apps. I am trying to implement a more "Swift-ier" version (using less obj-c, so the memory might be the issue here) in this app. The error is the title of this issue, and I know it has to be related to the camera intrinsic matrix but everything looks solid to me so I'm finally going to reach out for help.

This is how I am retrieving the intrinsic matrix of the camera:

        var matrix = matrix_float3x3.init()
        
        if let camData = CMGetAttachment(sampleBuffer, key: kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, attachmentModeOut: nil) as? Data {
            camData.withUnsafeBytes { (buffer: UnsafeRawBufferPointer) in
                guard let pointer = buffer.bindMemory(to: matrix_float3x3.self).baseAddress else {
                    return
                }
                matrix = pointer.pointee
            }
        }
        
        let px = matrix.columns.0.x
        let py = matrix.columns.1.y
        
        guard let uiImage = imageFromCVPixelBuffer(pixelBuffer: imagePixelBuffer) else { fatalError() }
        
        CVPixelBufferUnlockBaseAddress(imagePixelBuffer,[])
        
        let visp_detector = VISPDetector()
        visp_detector.detectTags(uiImage,
                                 cam_px: Double(px),
                                 cam_py: Double(py),
                                 tagSize: Double(0.0315))

The matrix looks like this:

| 1527.41357     0            539.5 |
| 0           1527.41357     959.5 |
| 0               0            1   |

It is a fairly simple implementation as follows:

    vpImage<unsigned char> I = [ImageConversion vpImageGrayFromUIImage:frame];
    
    float u0 = I.getWidth()/2;
    float v0 = I.getHeight()/2;
    
    if(cam_px == 0.0 && cam_py == 0.0){
        cam_px = 1515.0;
        cam_py = 1515.0;
    }
    
    float quadDecimate = 3.0;
    int nThreads = 1;
    detector.setAprilTagQuadDecimate(quadDecimate);
    detector.setAprilTagNbThreads(nThreads);
    
    vpCameraParameters cam;
    cam.initPersProjWithoutDistortion(cam_px,cam_py,u0,v0);
    std::vector<vpHomogeneousMatrix> cMo_vec;
    detector.detect(I, tagSize, cam, cMo_vec);

Where the parameters look fairly good to me:

cam_px	double	1526.1195068359375
cam_py	double	1526.1195068359375
u0	float	540
v0	float	960

[Deleted user]

Small but important update, this only occurs when a tag is in frame. If a tag is not in frame, detect() runs smoothly.

[Deleted user]

I tried a different purely objective-c++ approach and the algorithm still does not converge.

`- (NSArray *)detectTags:(CMSampleBufferRef)frame
cam_px:(double)cam_px
cam_py:(double)cam_py
tagSize:(double)tagSize
{

CVImageBufferRef buffer = CMSampleBufferGetImageBuffer(frame);
CVPixelBufferLockBaseAddress(buffer, 0);

void *baseAddressY = CVPixelBufferGetBaseAddressOfPlane(buffer, 0);
int bufferWidthY = (int)CVPixelBufferGetWidthOfPlane(buffer, 0);
int bufferHeightY = (int)CVPixelBufferGetHeightOfPlane(buffer, 0);
int bytePerRowY = (int)CVPixelBufferGetBytesPerRowOfPlane(buffer, 0);

/* create grayMat using obtained parameters */
cv::Mat grayMat = cv::Mat(bufferHeightY, bufferWidthY, CV_8UC1, baseAddressY, bytePerRowY);

vpImage<unsigned char> I;
vpImageConvert::convert(grayMat, I);
    
float u0 = I.getWidth()/2;
float v0 = I.getHeight()/2;

if(cam_px == 0.0 && cam_py == 0.0){
    cam_px = 1515.0;
    cam_py = 1515.0;
}

float quadDecimate = 3.0;
int nThreads = 1;
detector.setAprilTagQuadDecimate(quadDecimate);
detector.setAprilTagNbThreads(nThreads);

vpCameraParameters cam;
cam.initPersProjWithoutDistortion(cam_px,cam_py,u0,v0);
std::vector<vpHomogeneousMatrix> cMo_vec;
detector.detect(I, tagSize, cam, cMo_vec);
    
int tagNums = (int) detector.getNbObjects();
for(int i=0; i < tagNums; i++){
    
    std::vector<vpImagePoint> polygon = detector.getPolygon(i);
    vpImagePoint cog = detector.getCog(i);
    vpTranslationVector trans = cMo_vec[i].getTranslationVector();
    
}

CVPixelBufferUnlockBaseAddress(buffer, 0);


return nullptr;

}`

[Fabien Spindler]

Difficult to figure out what is wrong!

Some suggestions:

• The error occurs in svdLapack() that is called during pose estimation. You may try to see if the tag detection is working without estimating the pose, just to confirm that the pose algorithm is throwing the exception. To do that call simply detector.detect(I, tagSize, cam); and do not use cMo_vec
• If the previous test is working without throwing an exception, try to change the pose estimation algorithm using setAprilTagPoseEstimationMethod() or directly in the vpDetectorAprilTagconstructor
• If Eigen or OpenCV 3rd parties available, disable Lapack usage cmake ../visp -DUSE_BLAS/LAPACK=OFF -DUSE_EIGEN3=ON; make and try again (similar with OpenCV)

[Deleted user]

bool detect(const vpImage<unsigned char> &I); bool detect(const vpImage<unsigned char> &I, double tagSize, const vpCameraParameters &cam, std::vector<vpHomogeneousMatrix> &cMo_vec, std::vector<vpHomogeneousMatrix> *cMo_vec2=NULL, std::vector<double> *projErrors=NULL, std::vector<double> *projErrors2=NULL);

cMo_vec is not an optional parameter, so I'm not sure how I could execute that test. Correct me if wrong, on VISP 3.5.0

[Deleted user]

Either way, the situation has magically resolved itself by reimporting VISP. I did reimport it several times over the past few days, but finally decided to just redownload and reimport and that one worked. Very odd behavior, something had to be corrupted.