Optimization crash during `extrinsic_refine` step

[Jun Zhang]

Hi there,

I am following the calibration steps with the provided dataset ('scene2') and ran into some branch in the extrinsic_refine step .
Could you please share some thoughts on the issue? I have tried the scene1 and got the same issue.

Thanks

roslaunch mlcc extrinsic_refine.launch

process[extrinsic_refine-1]: started with pid [945409]
process[rviz-2]: started with pid [945410]
---------------------
iteration 0
extrinsic_refine: /home/cyngn/github/catkin_mlcc_ws/src/mlcc/include/extrinsic_refine.hpp:178: void EXTRIN_OPTIMIZER::optimize(): Assertion `!std::isnan(residual2)' failed.

[Jun Zhang]

The problem comes from here
It looks like we have an underdetermined system.

cv::solve(matA, matB, matX, cv::DECOMP_QR);

matA: 
[4668.396019965554, 762.2615729792876, 443.5128503180619, 25.7672809111645, -50.61608196270411, -170.2821341374004, 0, 0, 0, 0, 0, 0;
 762.2615729792853, 5517.516799518293, 331.9476116036997, 13.71723803017447, 36.60771030027582, -285.2359537322813, 0, 0, 0, 0, 0, 0;
 443.5128503180659, 331.9476116037035, 1141.239173941337, 57.1814471478689, 14.49662191019416, -18.02984394798824, 0, 0, 0, 0, 0, 0;
 25.76728091116443, 13.71723803017436, 57.18144714786875, 4.913689564802027, 1.673394500703981, 0.1322514847335978, 0, 0, 0, 0, 0, 0;
 -50.61608196270406, 36.60771030027578, 14.49662191019422, 1.673394500703978, 5.185042253110018, -0.7509567169207, 0, 0, 0, 0, 0, 0;
 -170.2821341374004, -285.2359537322811, -18.02984394798815, 0.1322514847335943, -0.7509567169207013, 24.62999979983148, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
matB: 
[20.99431487687908;
 90.8448330263231;
 -0.971213307257915;
 1.655736040733197;
 -0.4554962466367958;
 -6.108141842074613;
 -0;
 -0;
 -0;
 -0;
 -0;
 -0]
matX: 
[-nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan;
 -nan]

[Xiyuan Liu]

Hi @jun-zhang-cyngn, sorry for my late reply. I am not sure if you have solved this issue. In our latest project here, we choose Eigen to solve the AX=b function. Hope this helps.

[yucong lin]

I came across the same issue.

[yucong lin]

yuconglin@e9b0ab1 I fixed in my forked branch. At least it can run through now. It looks like current main branch only support calibrating a single LiDAR's extrinsic.

[Xiyuan Liu]

yuconglin@e9b0ab1 I fixed in my forked branch. At least it can run through now. It looks like current main branch only support calibrating a single LiDAR's extrinsic.

Hi @yuconglin, our extrinsic_refine currently only supports calibrating one extrinsic at once, so if you have multiple LiDARs, you need to execute this launch file multiple times. But our global_refine accepts multiple extrinsics so that you could copy and paste them all together.

[yucong lin]

Thanks for replying. I figured it out too. Thanks again for making the repo public. I learned a lot.