TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.

Home Page:https://voca.is.tue.mpg.de/en

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to control expression in the edit_sequence.py?

jamja11 opened this issue · comments

Thanks for share of the great work!
I want to know how to control variance of expression from flame model.
and I want to change smile obj
so I found.. alter_sequence_shape within the edit_sequence.py, the model.betas[300:400] this can change the expression.
I changed parameters.. but I don't know mouth control..
please help and thank you

Indeed, changing betas 300-400 can change the expression but it will mess up the speech-related expressions. For expression control, one would need to disentangle the speech- and emotion-related expression to change only the emotional content while keeping the speech-related content unchanged. Unfortunately, we don't have any solution for this yet.

Thank you for your response!
now, I don't use speech.. umm I just want to controll expression like webdemo
https://flame.is.tue.mpg.de/interactivemodelviewer.html
image

I try to edit model.betas[300:400] =
[3., 2., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0.]
image
umm.. nose shape is strange..
I want to know nose variation controll...
Thank you