【QUESTION】The necessity of convert SMPL parameter to STAR

[NERO-S]

hello，thanks to the wonderful works！
now, I have a task to regress SMPL parameters from image, so the annotations are SMPL groundtruth, and I use SMPL module to get human mesh vertices and then regress 3d joints location from human mesh. I wonder to know if I want to replace SMPL module to STAR, do I need run the covert script to convert SMPL annotations to STAR, and use the new annotations as supervision. Does the convert process is necessary? Because STAR and SMPL are both use 10 dimension beta and 72 dimension pose parameters, and I use original SMPL parameter as input and feed into STAR model, I got similar result human mesh but STAR result's human mesh shape a more accurate, but pose are similar. Can not distinguish the differences of pose between them by just watching.

[Ahmed A. A. Osman]

Hi @NERO-S, Thanks for the question, we get asked this a lot, I'll write a blog about it.

First, what are your SMPL annotations, are they 2D joint locations (landmarks) or pose and shape parameters? If they are 2D joints, STAR has an identical kinematic tree of SMPL so you can fit the model to those 2D joints.

However, If they are pose and shape parameters then the conversion is necessary. There is a slight difference between STAR and SMPL in the pose space, and a much bigger difference in the shape space, as we retrained the model from scratch on more data. For example, if you train a neural network to regress the SMPL (pose and shape parameters), but at test time you remove SMPL and add STAR you'll be getting a different body (the shape will differ certainly), and your test error will be probably a lot higher.

I am going to close this issue now, feel free to re-open if this is not clear.

Ahmed