How to use CPF on both hands?

Question

How to use CPF on both hands?

buaacyw opened this issue 3 years ago · comments

Thanks a lot for your great work!
I have a question:
Since you only use the MANO_RIGHT.pkl, it seems that CPF currently can only construct right hand model, right?
What is needed to be modified to use CPF on both hands?
Thanks!

kelvin34501 · Answer 1 · Mon May 24 2021 22:45:17 GMT+0800 (China Standard Time)

Since you only use the MANO_RIGHT.pkl, it seems that CPF currently can only construct right hand model, right?

Yes. CPF uses FPHAB and HO3D. They only have annotations for right hands.

What is needed to be modified to use CPF on both hands?

Hand region assignment needs to be regenerated.
Hand anchors needs to be regenerated.
ManoLayer theoretically does not need to be modified. File a bug if you encounter weird results on left hands.
HoNet and PICR model needs to be retrained on datasets that contain annotations for both hands.

kelvin34501 · Answer 2 · Mon May 24 2021 22:53:12 GMT+0800 (China Standard Time)

We will release scripts for generating hand region assignment and anchors when we manage to get our work published.

Lixin YANG (杨理欣) · Answer 3 · Mon May 24 2021 23:07:47 GMT+0800 (China Standard Time)

@buaacyw Hi, Yiwen, Thanks for your interest in our work!

Currently, we do not plan on retraining MIHO on any both-hand dataset.
But we will schedule to develop the anatomically constrained MANO on both hands in our project manotorch