lixiny / CPF

[ICCV 2021] CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

what's the meaning of "adapt"?

haolyuan opened this issue · comments

I notice that there are hand_pose_axisang_adapt_np and hand_pose_axisang_np in your code. Could you please explain what's the difference between them?

For code here:

hand_pose_axisang_adapt_np = np.asarray(test_sample[CIAdaptQueries.HAND_POSE])
hand_pose_axisang_np = np.asarray(test_sample[CIDumpedQueries.HAND_POSE])

hand_pose_axisang_adapt_np is the pose from the dataset's annotation (name coming from CIAdaptQueries which stands for an adaptor of the hodataset);
hand_pose_axisang_np is the pose from the dump results.

Can I get hand_joints_adapt_np by using hand_pose_axisang_adapt_np? When I run demo, I found they are not the same.I have take global trans into consideration.

One more question! I would like to know what's the meaning of b_axis = (transf[:, 1:, :3, :3].transpose(2, 3) @ b_axis.unsqueeze(-1)).squeeze(-1) in axislayer.py. I know the meaning of b_axis = hand_joints[:, self.joints_mapping] - hand_joints[:, [i + 1 for i in self.joints_mapping]] is the relative jts vector in mano order.

Can I get hand_joints_adapt_np by using hand_pose_axisang_adapt_np? When I run demo, I found they are not the same.I have take global trans into consideration.

It depends on the dataset. hand_joints_adapt_np and hand_pose_axisang_adapt_np are annotations from dataset. However, different datasets might use different joint locations. You might need to try ManoAdaptor to see whether you can get the annotated joints.

One more question! I would like to know what's the meaning of b_axis = (transf[:, 1:, :3, :3].transpose(2, 3) @ b_axis.unsqueeze(-1)).squeeze(-1) in axislayer.py. I know the meaning of b_axis = hand_joints[:, self.joints_mapping] - hand_joints[:, [i + 1 for i in self.joints_mapping]] is the relative jts vector in mano order.

The mano pose is in its local frame, so it needs to be transformed to get the axis in global frame.

I almost understand. But there is still a question. We know that the mano skeleton goes like J'=R0*R1*...*Rp(J-Jp)+Jp', where p mean the closest parent. So I don't understand what's the meaning of RpT*(J'-Jp'), where T means transpose. I think there must be something wrong about my understanding. Thanks a lot!