Workflow to generate custom pose (e.g. from Mixamo fbx)
huh8686 opened this issue · comments
Hi.
Could you tell how the sequence pose files(located in ./eval_constant/sequences/ *.npy) are generated ??
It seems like those files were converted from Mixamo files but couldn't find a clue to create those.
If you don't mind , could you let us know this process ?
Hi, you may find the description for the pose files here. As for the Mixamo converting, you may download the Mixamo fbx and export the motion as BVH using Blender. Then organize the rotation of each joint (you may need to build up the correspondence manually) following the pose file definition. That's it!
Hi. Firstly, thanks for sharing this great work. I tried to regenerate the results with mixamo domain animations. Since there is no info about the pose data preparation, I was trying to work around the solution for one of the pose files in the repo. I found the original animation file for greeting.npy on mixamo and tried to regenerate the same values for this given file. I believe I generated the pose files with axes matching (had to manually find the best fit). However, when i used the same animation pose file which i generated, I wasn't able to get similar results, even the rotations were 1/3 of the whole movement (for example for arm flexing).
I checked the other issues and your replies for how to prepare custom pose files but it seemed like I wasn't able to imitate the greeting.npy pose file from mixamo fbx. (used blender .bvh file to parse the angles etc.)
If it's possible, could you explain how to generate custom pose files more precisely?
(P.S. I am too hyped to make this work but already spent quite a long time to generate pose files, so I frankly appreciate if you could give some kind of detailed guidance) . Thanks :)
Hi, from your description I think you are doing the right thing. It will be helpful if you can show me a short video of your result. I would suggest you to check, without seeing your result, that if you make the conversion of euler angles (in the bvh file) to the rotation vector (in the npy file) correctly and check if the mapping from Mixamo skeleton to SMPL skeleton is correct.
Hi, thanks for the reply!
So what I did was;
1-) Download "StandingGreeting.fbx" from mixamo for 'maynard' character.
2-) I used Blender to export '.bvh' file from this fbx by this settings:
3-) I found bvh parser python library (https://github.com/olafhaag/bvh-toolbox/)
4-) By using function get_euler_angles the bvh library, I extracted the joint rotations in degrees, then convert them to radians.
5-) Then I imported both your repo's result and my result on blender and adjusted the axes on the .npy file which I generated with steps above.
6-) Result was like this;
The character on the right is the result I got after passing rotation angles which I generated. Even though, the rotation angles somewhat should be related to the bind pose (T-pose in this case), my generated character was always on T-pose at the start frame of the generated animation.
--- I also considered your suggestion to align the input character before generating the BVH file from Blender by applying some value of Z axis translation on Transformation - > Location section. The BVH file after aligning the character yielded good results however, still there was some problem with the animation and the result looks like this:
result.mp4
I had worked on adjusting the axes to each other and also factor one axis to match to the original animation but on elbow I always got this massive rolling with respect to bone axis.
It seems that converting from Euler angles to rotation vector is not working properly.
Hi, Thanks for providing the source code. I also downloaded and tried to process the greeting.fbx file. It seems the root location in the bvh file is OFFSET -0.000008 98.962692 3.219785. However, the root location in the npy file is 0.004516, -0.005117, 0.004639, could please let me know how did you get the root location? Thanks!