Unable to create Constant transitions
AGameWriter opened this issue · comments
Summary
As far as I can tell, the plugin seems to be working correctly as designed, but in previous version I was somehow able to work the settings so that my keyframes would be Constant and would transition instantly instead of blending. I'm wondering if that got written out in 1.3.1?
If you're doing a traditional 3D animation, blending lip shapes makes perfect sense. I happen to be using 2D sprite based lips, a la Southpark or Lego Movie.
I've tried setting up all of the keyframes in my poses as Constant keys, but if I Bake the resultant Action, using Visual Keying and Clean Curves, I still get stepped interpolation.
Am I blind? I know some parts of the interface were streamlined, but I know I managed to get everything working previously.
SIDE QUESTION: Is there a forum or Discord server somewhere we can discuss the piddly "how do I?" questions about using this plugin as it is developed? I feel really dumb opening a bug report for what I suspect is not actually a bug.
The resulant baked curves:
Rhubarb Lipsync NG addon version
1.3.1
Blender Version
4.1.1
Operating System
Windows 10
Indeed I was not thinking much about the 2D cases where interpolation makes no sense. Thanks for bringing this up. I will add some more NLA baking options back to cover this use case better in the future.
Can you try to disable the blending of all the NLA Strips before you bake them to a new Action? You need to use the Alt+click trick:
It looks to me once there is no blending on the Strips the keyframes has only constant changes. You can then also change the interpolation to constant too with this same trick.
SIDE QUESTION: Is there a forum or Discord server somewhere we can discuss the piddly "how do I?" questions about using this plugin as it is developed? I feel really dumb opening a bug report for what I suspect is not actually a bug.
Good point. There is currently only bug/feature request ticket templates. I guess I'll add another general support template. Perhaps that would. Note often a bug/flaw is no in the code but in the design :-D
Sorry it's taken me so long to get back to you. The Alt-Click trick you mentioned does seem to have worked, although of cours it is an extra step for 2D work.
It still took a lot of post-baking adjustments to get things looking right. Part of that was timing, and part of that was errant mouth shapes. I'm sure part of that is the background noise in my audio sample, but I'm wondering if the transitions have an influence on the timing.
Either way, this not-bug can be closed. Thanks!
PNGS0001-0270.webm
@AGameWriter In the latest version, there is now a No Blending option in the Strip Placement section. Using this together with a single track should ease 2D animation setup.
Regarding the detection accuracy, that is an inherent problem of the underlying Rhubarb engine, which is quite dated. You can try to add the dialog file (transcript). Some people claim better results while using this:
https://github.com/DanielSWolf/rhubarb-lip-sync?tab=readme-ov-file#basic-command-line-options :
With this option, you can provide Rhubarb Lip Sync with the dialog text to get more reliable results. Specify the path to a plain-text file (in ASCII or UTF-8 format) containing the dialog contained in the audio file. Rhubarb Lip Sync will still perform word recognition internally, but it will prefer words and phrases that occur in the dialog file. This leads to better recognition results and thus more reliable animation.
For instance, let’s say you’re recording dialog for a computer game. The script says: “That’s all gobbledygook to me.” But actually, the voice artist ends up saying “That’s just gobbledygook to me,” deviating from the dialog. If you specify a dialog file with the original line (“That’s all gobbledygook to me”), this will still allow Rhubarb Lip Sync to produce better results, because it will watch out for the uncommon word “gobbledygook”. Rhubarb Lip Sync will ignore the dialog file where it audibly differs from the recording, and benefit from it where it matches.
It is always a good idea to specify the dialog text. This will usually lead to more reliable mouth animation, even if the text is not completely accurate.
The dialog file:
If you are looking for better accuracy, I saw a new lipsync addon pop up on BlenderMarket a few months ago, which uses the open Whisper AI model. It has better accuracy, especially for non-English languages.