weird result in the example data

[Baldr_Yao]

Hi, @Juyong and @yaoyx689 , thanks for kindly sharing the source code. There are so many brilliant ideas and the results are stunning!

However, when i run the code, there some weried results.
I run this code at the example data "partial overlap" , mesh_0018.obj to mesh_0020.obj, which shown as follows:

1. mesh_0018.obj:
   

2. mesh_0020.obj
   

3. the result
   

4. the provided result
   

I run the code utilize the code under src_cvpr dir, which utilizes the nearest point as the initial correspondence as defaults.
could you tell me why I generate a weird result?
Thanks in advance, your reply will vary appreciated.

[Yuxin Yao]

Hi, setting "paras.pruning_type = SIMPLE;" in "main.cpp" can get good result. Because the initial difference between the source and the target surface is relatively large, we get initial corresponding points using closest point pairs with simple rejection criteria(distance and normal). Please refer to the implementation details in the article. Notice that our method will use rejection in the first iteration and RPTS/NICP will use rejection in all iterations when setting "paras.pruning_type = SIMPLE;", this is determined by the setting of the objective function in each method.

[Baldr_Yao]

Hi, setting "paras.pruning_type = SIMPLE;" in "main.cpp" can get good result. Because the initial difference between the source and the target surface is relatively large, we get initial corresponding points using closest point pairs with simple rejection criteria(distance and normal). Please refer to the implementation details in the article. Notice that our method will use rejection in the first iteration and RPTS/NICP will use rejection in all iterations when setting "paras.pruning_type = SIMPLE;", this is determined by the setting of the objective function in each method.

Thanks for your reply, I really appreciate it and it works.

However, with this setting, I find limbs are may not well aligned sometimes, as follows. Is there any solution for this issue?

reuslt:

ground-truth:

[Yuxin Yao]

Could you tell me which source model you use? Did you register from 'mesh_0028.obj' to 'mesh_0030.obj'? When using the closest point as the correspondences, it's difficult to deal with the model with the big difference between two surfaces(eg. the angle of arm deformation exceeds 90°).

[Baldr_Yao]

Could you tell me which source model you use? Did you register from 'mesh_0028.obj' to 'mesh_0030.obj'? When using the closest point as the correspondences, it's difficult to deal with the model with the big difference between two surfaces(eg. the angle of arm deformation exceeds 90°).

Thanks for your reply.
I register from 'mesh_0018.obj' to 'mesh_0028.obj' with Fast RNRR. And you are right, it's difficult to deal with the model with the big difference between two surface.
I wonder if there are ways to avoid this issue? I guess correct initial correspondences may be helpful, and if I want to make initial correspondences by hand, is there any tool ? Or there are more smarter ways to make correct initial correspondences ? (I also try with SHOT, the result is still unsatisfactory)

[Yuxin Yao]

yes, you are right, it really needs some good initial correspondences.

If you have some initial correspondences, you can set the "paras.corres_type =LANDMARK" and "paras.pruning_type=NONE" in "main.cpp" and run
$./Fast_RNRR <src_file> <tar_file> <out_file> 3 <landmark_file> (refer to readme.md)

In the old version, the way we use landmarks is to directly use the input landmarks as the correspondences for the first iteration, and still use the closest point for other iterations.
In the new updated version, we use the landmarks as an energy term to participate in the optimization of each iteration, similar to NICP(Optimal Step Nonrigid ICP Algorithms for Surface Registration), you can refer to it for details if needed.

Initial correspondences: you can manually mark some corresponding points using rendering tools, such as 'blender' or directly see the label of the point in 'meshlab'; Because the SHOT feature is still ambiguous when matching, especially in the part of non-rigid deformation, it's not accurate. You can also try to modify the parameters of SHOT to see if the result can be better (you can modify them in 'CmdLineParams' (143 lines in 'tools.h'). They are consistent with the parameters in SHOT's source code). In my opinion, these hand-designed local descriptors are inevitably ambiguous and inaccurate.

It's worth noting that when the number of landmarks is much smaller than the number of points on the source surface, you need to reduce the 'paras.alpha' and 'paras.beta' in 'main.cpp' to make the landmarks useful.

[Baldr_Yao]

yes, you are right, it really needs some good initial correspondences.

If you have some initial correspondences, you can set the "paras.corres_type =LANDMARK" and "paras.pruning_type=NONE" in "main.cpp" and run
$./Fast_RNRR <src_file> <tar_file> <out_file> 3 <landmark_file> (refer to readme.md)

In the old version, the way we use landmarks is to directly use the input landmarks as the correspondences for the first iteration, and still use the closest point for other iterations.
In the new updated version, we use the landmarks as an energy term to participate in the optimization of each iteration, similar to NICP(Optimal Step Nonrigid ICP Algorithms for Surface Registration), you can refer to it for details if needed.

Initial correspondences: you can manually mark some corresponding points using rendering tools, such as 'blender' or directly see the label of the point in 'meshlab'; Because the SHOT feature is still ambiguous when matching, especially in the part of non-rigid deformation, it's not accurate. You can also try to modify the parameters of SHOT to see if the result can be better (you can modify them in 'CmdLineParams' (143 lines in 'tools.h'). They are consistent with the parameters in SHOT's source code). In my opinion, these hand-designed local descriptors are inevitably ambiguous and inaccurate.

It's worth noting that when the number of landmarks is much smaller than the number of points on the source surface, you need to reduce the 'paras.alpha' and 'paras.beta' in 'main.cpp' to make the landmarks useful.

It is really helpful, thanks for your responsible reply.
I will try the mentioned strategies, and here I will close this issue.