Database of STAR pose/shape parameters

Question

Database of STAR pose/shape parameters

ThibaultGROUEIX opened this issue 3 years ago · comments

Hi,

Thanks for your awesome work. :)
I would like to train an ML task using the STAR model. Do you know what is the biggest / most diverse set of pose and shape parameters for the STAR model available online?

Thanks a lot for your help,
Thibault

ThibaultGROUEIX · Answer 1 · Mon Feb 07 2022 18:53:45 GMT+0800 (China Standard Time)

Hi @ahmedosman ,

I would really appreciate it if you could follow-up on this. Is there a database of STAR parameters with diverse shapes and poses?

Best regards,
Thibault

Ahmed A. A. Osman · Answer 2 · Tue Feb 08 2022 17:07:44 GMT+0800 (China Standard Time)

HI @ThibaultGROUEIX ,

Sorry for my late response.

Pose:

The kinematic tree (rig) of STAR is identical to SMPL, so to get a diverse set of body poses we could use the AMASS dataset (https://amass.is.tue.mpg.de/) to get the pose parameters - directly plugging in the pose parameters into STAR.

Shape:

For the shape, we can simulate diverse body shapes by sampling the betas of STAR (shape parameters) from a Gaussian distribution, because each of the principal components in the shape space is normalized such that +/- 1 beta corresponds to +/-1 standard deviation. The first PC components correspond to the training subject's heights, so +/- 4 beta for the first PC covers more than 99% of the variance of the heights of the subject in the CAESAR and SizeUSA dataset. The second PC components correspond to the subject BMI. Varying +/-3 standard deviation of the first two PC already covers variation in subject height and weight.

In summary, poses from AMASS and sampling the betas from a univariate Gaussian. Feel free to reach out via email if something is not clear or further questions (firstname.lastname@tuebingen.mpg.de) . Can I close the issue now?

ThibaultGROUEIX · Answer 3 · Wed Feb 09 2022 16:27:08 GMT+0800 (China Standard Time)

Awesome answer, thanks a lot ! Thibault

ThibaultGROUEIX · Answer 4 · Mon Feb 14 2022 20:47:36 GMT+0800 (China Standard Time)

One last question on this topic if you don't mind.
Why optimize the pose parameters in the conversion script from SMPL to STAR convert_smpl_2_star if the kinematic tree is the same?
in https://github.com/ahmedosman/STAR/blob/bc6aa8d9be1192ce90d394fdf8898347ec31fe39/star/convertors/losses.py#L67
Why not copy the SMPL pose and just optimize the shape parameters?
Thanks for your answer,
Best,
Thibault

Ahmed A. A. Osman · Answer 5 · Mon Feb 14 2022 22:23:56 GMT+0800 (China Standard Time)

@ThibaultGROUEIX

the SMPL and STAR pose space (kinematic tree and pose deformations) are similar but not exactly identical, there are differences:

The joint regressors are not identical, it was retrained for STAR. So if we overlay SMPL/STAR joints their 3D position are close, but not exactly identical.
The pose corrective deformation functions are different, for STAR the deformations are spatially local and sparse and also we condition on body shape.

As a result of 1 and 2, if you plug pose parameters in both SMPL/STAR the meshes wouldn't be exactly identical, but similar to some extent for most of the poses. So if you drive STAR with SMPL pose parameters from AMASS the motions will be plausible.

In the conversion script, I included the pose/shape parameters if the user is interested in getting a very similar STAR mesh to a given SMPL mesh.

Does this make sense?

best,
Ahmed

ThibaultGROUEIX · Answer 6 · Tue Feb 15 2022 16:22:01 GMT+0800 (China Standard Time)

Thanks Ahmed, I love your detailed answers.

Tal Barami · Answer 7 · Wed Apr 27 2022 21:46:29 GMT+0800 (China Standard Time)

HI @ThibaultGROUEIX ,

Sorry for my late response.

Pose:

* The kinematic tree (rig) of STAR is identical to SMPL, so to get a diverse set of body poses we could use the AMASS dataset  (https://amass.is.tue.mpg.de/) to get the pose parameters - directly plugging in the pose parameters into STAR.

Shape:

* For the shape, we can simulate diverse body shapes by sampling the betas of STAR (shape parameters) from a Gaussian distribution, because each of the principal components in the shape space is normalized such that +/- 1 beta corresponds to +/-1 standard deviation. The first PC components correspond to the training subject's heights, so +/- 4 beta for the first PC covers more than 99% of the variance of the heights of the subject in the CAESAR and SizeUSA dataset. The second PC components correspond to the subject BMI. Varying +/-3 standard deviation of the first two PC already covers variation in subject height and weight.

In summary, poses from AMASS and sampling the betas from a univariate Gaussian. Feel free to reach out via email if something is not clear or further questions (firstname.lastname@tuebingen.mpg.de) . Can I close the issue now?

Hey Ahmed and thanks for all of your replies in this thread.
I was wondering if you could elaborate more on the pose parameters and how to use AMASS to receive them.
I'm trying to generate my own dataset through STAR and wonder how to sample the pose code.

Thanks,
Tal

Ahmed A. A. Osman · Answer 8 · Mon May 02 2022 17:46:12 GMT+0800 (China Standard Time)

Hi @TalBarami ,

Thanks for your interest in STAR. AMASS uses the SMPL model, so the datasets provide pose and shape parameters for SMPL. If we plug in the pose parameters in SMPL we will get meaningful motions - there should plenty of demos on how to use AMASS parameters to animate SMPL. STAR is a similar model to SMPL but not identical as I mention above, however if you simply use the exact same AMASS pose parameters in STAR you'll get meaningful motions (the meshes won't be identical though).

Thanks,
Ahmed