kwea123 / VTuber_Unity

Use Unity 3D character and Python deep learning algorithms to stream as a VTuber!

Home Page:https://www.youtube.com/playlist?list=PLDV2CyUo4q-JFGrpG595jMdWZLwYOnu4p

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Model moves erratically when connected with python

astroty opened this issue · comments

commented

ezgif-6-4c079af072f6
When connected to the demo.py through --connect (gpu or cpu) the model starts moving her head uncontrollably. In the presented gif it's moving slower than it should be due to recording lag, so it's worse than shown

Hardware used is an i7 5960x with a GTX 980

same thing xP

Ubuntu 20.10

Me too

Windows 10, RTX 2070, i9-10900

Jup same here with --cpu mode

Arch (Kernel 5.10.16-arch1-1), Ryzen 5 2600

Same here in --cpu mode

Windows 10, Intel Core i3-6098P

Spanish Language

I think I found the culprit for this.
On the debugging window from demo.py the whole alignment to my face always looked fine.
So I downloaded the unitychanproject.zip und dug around there.
In the file UnityChanControlScriptWithRgidBody.cs we recieve the data that demo.py sends.
The culprit there is the float.Parse() function which might work different depending if you're system is configured that your decimal separator is a comma "," or a point "."
So the relatively small values with the wrong decimal separator seem to get interpreted as bigger than they actually are because it just seem to skip the decimal separator that isn't defined as the current default and ignore any leading zeros as well.
So the strings "-11.4411" and "0.0548" suddenly become -114411 and 548 instead.

the lines 72-77 where float.Parse() appears:

roll = float.Parse(res[0])*0.4f + roll*0.6f;
pitch = float.Parse(res[1])*0.4f + pitch*0.6f;
yaw = float.Parse(res[2])*0.4f + yaw*0.6f;
min_ear = float.Parse(res[3]);
mar = float.Parse(res[4])*0.4f + mar*0.6f;
mdst = float.Parse(res[5]);

To fix that I just also gave float.Parse the "CultureInfo.InvariantCulture" argument so it is forced to interpret points as the decimal separator when your default culcture (in my case German) uses another decimal seperator

fixed lines 72-77 of UnityChanControlScriptWithRgidBody.cs:

roll = float.Parse(res[0], CultureInfo.InvariantCulture)*0.4f + roll*0.6f;
pitch = float.Parse(res[1], CultureInfo.InvariantCulture)*0.4f + pitch*0.6f;
yaw = float.Parse(res[2], CultureInfo.InvariantCulture)*0.4f + yaw*0.6f;
min_ear = float.Parse(res[3], CultureInfo.InvariantCulture);
mar = float.Parse(res[4], CultureInfo.InvariantCulture)*0.4f + mar*0.6f;
mdst = float.Parse(res[5], CultureInfo.InvariantCulture);

This also needs another dependency:

using System.Globalization;

Haven't checked yet if this fix will create problems if the default culture uses a point as their decimal separator.
At least this fixes it for me. Sadly this part of the project isn't actually in this repository or in it's own repository. (if it is I couldn't find it).