Import: Temperance should be off by default
lyuma opened this issue · comments
Describe the bug
Guess Original Bind Pose + Temperance changes data from the intent of the gltf file. For this reason, It should be off by default.
For example, on some models, it will flip the hip bone upside down upon re-import. This behavior is not good by default. In the case where the user is importing a model with unusual bone poses, they can choose to select the box when importing. This matches the behavior of FBX.
To Reproduce
Steps to reproduce the behavior:
- File -> Import ->glTF 2.0
- Select xbot.glb
- Leave default settings "Guess Original Bind Pose" checked and Temperance.
- click on Armature. Select Square tab -> Viewport Display -> In Front
- Notice that hips bone is pointing down.
- Repeat, but uncheck "Guess Original Bind Pose".
Expected behavior
The default should use the armature exactly as specified in the glb file.
The user is still free to change the import setting based on their preferences.
Screenshots
Left = Blender | Right = Temperance
.blend file/ .gltf
xbot.gltf.zip
Version
- OS: Windows 10 / 10.0.19044.1706
- Blender Version: Both 2.93.0 and 3.2 Beta
Additional context
Around version 2.83, blender released a glTF importer with Guess Original Bind Pose and Temperance setting enabled by default: added in 2020 by @scurest in #946.
I understand this bug report may be controversial. I'm hoping to see some discussion on the pro's and con's of using various import settings. The reason I'm opening this is because I do not like to see data degradation over time due to data loss on re-import (in this case, the original bone orientation being lost).
See also: Github issue in VRM specification vrm-c/vrm-specification#34 about discarding local axis.
See also; a discussion on normalization of bone direction in the context of VRM by @TokageItLab: https://qiita.com/TokageItLab/items/e5880123a9f508b2769d#blender%E3%81%AEgltfimporter%E3%81%AB%E3%81%8A%E3%81%91%E3%82%8B%E6%8E%A8%E5%AE%9A (Japanese)
See also: https://qiita.com/TokageItLab/items/e5880123a9f508b2769d (Japanese)
I really wish I had filed this two years ago, but I didn't know enough then. I also wish I had filed this last year, but I guess better late than never.
Thanks!
~Lyuma
Also, the enum value BoneDir = Blender is difficult to understand and not good. It should be something like BoneDirEstimation = None.
Or there should place a checkbox BoneDirEstimation: bool and when it is true, it should allow the selection of Temperance and Fortune, then Temperance can be made the default.
My preference is for defaults that are usually mediocre over defaults that are sometimes great but sometimes catastrophic. "Blender" can be very, very bad.
Also, the enum value
BoneDir = Blenderis difficult to understand and not good. It should be something likeBoneDirEstimation = None.
There is no BoneDirEstimation = None. The conversion from glTF to Blender space is (X, Y, Z) -> (X, -Z, Y). Bones point along the (Blender +Y axis) so if there were no change, logically the (glTF -Z axis) would becomes the bone axis. But the "Blender" option makes the (glTF +Y axis) the bone axis because for whatever reason that's where the exporter happens to put it.
There really can't be a BoneDirEstimation = None IMO, since the bone dir concept don't exist in glTF, so even if you "do nothing", you've implicitly decided the (glTF -Z axis) should be the bone axis.
Although space conversion is performed, it is applied to the entire model, not just the bones, so you do not need to be aware of it. Blender's world space is +Z up, but +Y up for bones.
Also, glTF models use node format, not bones, and do not have a specific roll axis; a model with a roll axis set to +Y in blender will still have a roll axis of +Y in glTF, and will not have a roll axis of -Z in glTF.
The most problem is that the default estimation make them often destroy the original data without knowing that it is the direction of the bone that is being estimated.
Most people do not need to understand the conversion between glTF's Node format and Blender's Bone format, but we absolutely must warn them that they are destroying the original bone direction.
Personally if I'm handed a random glTF, I'd like a semi-usable skeleton and I don't care at all which axis of a bone was originally which.
Isn't the glTF format to allow for interoperability? It is a shame that this implicit estimation makes data as corrupt and destructive. The direction estimation may be able to estimate to some extent with respect to the roll axis, but it will certainly break the Bend axis.
As you can see at the top of this issue, certain bones will be broken, and bones that take a long time to set up, like fingers, will be corrupted. At least BoneDir=Blender does not corrupt the data, just it has the different space axes basically. It can just manually rotate all the EditBones by 90 deg * n value to make the correct conversion.
I do not deny the existence of estimation, but it should not be the default without caution "it is destructive".
You can't just rotate all bones by some constant after import. For one thing it breaks all animations. For another, there is not guaranteed to be a convention that holds for all bones. AFAIU Autodesk stuff in particular has a per-bone "direction", where often the "correct" axis is opposite on opposite sides of the model.
Certainly, when animation and mirroring are taken into account, manual correction may be difficult, but I see no reason why estimation should be the default. You said there is no BoneDirEstimation = None, but at least it is a "conversion" and not an "estimation". Still I believe that estimation should not be implicit in it.
The reason is the picture in my first post. "Blender" often results in every single bone pointing in the wrong directions.
Again, it is not wrong, it just does not adapt to Blender's bone system.
It may be inconvenient, but the Head and Tail do not always need to be connected in Blender. In fact, wrists with multiple children, and eyes which do not need to be connected are not connected basically.
Also, Blender only makes the bone Gizmo point to +Y, and if they want to have their own axis, they should have their own axis. It is your sophism that it is wrong if it does not always point from parent to child.
How is that relevant? All you've said is "Temperance" is not always correct. That's obvious. Neither is "Blender".
Like I said, "my preference is for defaults that are usually mediocre over defaults that are sometimes great but sometimes catastrophic".
One of the reasons I filed this bug is that you should be able to open a gltf file in Blender, make a few edits, and re-save it and use it in another application, without fear that the content of the document will be changed. (Yes, some corner cases exist where Blender splits a mesh+bone into two nodes, but this is equivalent). While a 1-to-1 conversion is fine, guessing logic can be arbitrary and irreversible.
However, for the case of skeleton bones, due to Temperance the exported glTF will have different bone directions than the original glTF, which may lose their original semantic meaning. For example, hips pointing up vs. hips pointing down, can cause failures in IK solvers.
The fact that Blender can corrupt bone hierarchies this way by default makes it difficult to work with others and debug problems, because a novice user may assume the default settings are safe.
Also, a case of Autodesk "direction" was brought up. This is not a glTF property, so that makes it a proprietary behavior of Autodesk. I have no problem with providing the option to fix Autodesk behavior, but this should not be default.
Neither is "Blender".
I disagree. The "Blender" setting causes the exported data to be the same as the imported data. This in my mind makes it correct.
Also, a case of Autodesk "direction" was brought up. This is not a glTF property, so that makes it a proprietary behavior of Autodesk. I have no problem with providing the option to fix Autodesk behavior, but this should not be default.
That's backwards, "Temperance" handles all glTFs exactly the same (with equal mediocrity). Such a glTF could come from anywhere. It is "Blender" which, as the name suggests, is hardcoded to the behavior of a specific program.
So which should win, roundtripping or genericity?
This repo does usually treat roundtripping as "correctness", that roundtripping should cause no change, or at least stabilize after one cycle (be idempotent). But importing does change many things. Another one is that skinned meshes are retargetted onto a certain pose (which may or may not be the same as the original one) at import time. Actually in the first place glTF is ideally an export-only format like JPEG. Despite roundtripping being the criterion for correctness, it's not actually something you should do, ie. it's kind of a special case, ie. not the default path.
There's also the question of what's being roundtripped, since things that don't exist in glTF aren't expected to be. Bone lengths are obviously completely lost. Bone axes have no special significance within glTF, if they have significance it has to be bestowed on them from outside, which again places you in the "having special knowledge about what you're doing" bin instead of the "don't care" default.
I'm reluctant to privilege files exported from Blender by default.
Temperance" handles all glTFs exactly the same (with equal mediocrity).
It is natural that importer uses the same algorithm for all glTFs, there is no reason Temperance should be default.
This repo does usually treat roundtripping as "correctness", that roundtripping should cause no change, or at least stabilize after one cycle (be idempotent).
The estimation Temperance and Fortune does not match input/output and may not work always. It lacks "correctness". As long as we use BoneDir = Blender, the inputs and outputs bone orientation match.
But importing does change many things. Another one is that skinned meshes are retargetted onto a certain pose (which may or may not be the same as the original one) at import time.
The estimation is causing changes to things that don't actually need to be changed without notifying it to users.
Actually in the first place glTF is ideally an export-only format like JPEG.
I hear that analogy often, but I always think it's a wrong analogy because the general 3D model format, not just the glTF format, is essentially similar to vector data.
To use an analogy to the problem with the current estimation being the default, it is like the default is to rasterize an SVG once, autotrace it, and save it as SVG format again.
Bone axes have no special significance within glTF
I disagree. For example, designer/rigger sets the axis to bend the skinned model's joint pretty, but it is corrupted by estimation.
In other words, a glTF model that rolls on the X axis and bends on the Y axis in Maya should ideally roll on the X axis and bend on the Y axis in other software basically (as long as the glTF output by Maya is so). BoneDir = Blender can do that within Blender as well.
The problem is that estimation creates an ambiguous rig specifically for Blender, which makes it incompatible with external applications. I don't think this is what glTF wants.
Game engines such as Unity are often an end-environment and thus to some extent are allowed to replace everything with their own rig system such as Humanoid, but Blender is a DCC software and you have to think about exporting the model.
You seem to be thinking of not exporting again the glTFs that you imported from outside of Blender for your 3D models.
I'm reluctant to privilege files exported from Blender by default.
On the contrary, it is the estimation that has given the exported glTF the privilege of being Blender specific.
The main reason for estimation on Blender is to improve the look of Gizmo. Even if Blender's bone Gizmo is not oriented from parent to child due to BoneDir = Blender, it is truly the correct data that faithfully emulates the glTF data. It just looks bad because Blender's bone Gizmo is a special one that always point +Y, but nothing is broken. So, It should not be the default to destroy the original bone data by the estimation just to make Gizmo look neat.
I don't think I can add anything that hasn't already been said, but I agree with @scurest. Estimation of a user-friendly armature is a better default for most users. The users for whom it's not the right default are more likely to understand skinning data well enough to find and disable the option. At the end of the day this is the default value of a single checkbox.
Worst the temperance default is fine, but in that case it needs to be made known by a popup, red text, etc. that it destroys the original data. Lack of notification means that most users will not be able to recognize if there is a bug caused by estimated data.
Furthermore, the armature's Automatic Bone Orientation and Find Bone Chains option is off by default in fbx and collada imports as well. That indicates that the glTF importer's default which destroy original data and do estimation is in the minority.
To chime in here since I recently ran into this:
- current behaviour does not allow for glTF roundtrip AND nice editing in Blender
- user has to choose between "bones look right in Blender but will export wrong / broken roundtrip" and "bones look wrong in Blender but roundtrip works".
There's two solutions that I can see beyond simply making "Blender" the default instead of "Temperance":
- hard and outside this plugin: allow Blender bones to be displayed looking in other directions. E.g. 3ds max has the ability to specify in the display settings of bones which direction is "forward".
- easier – allow converting back on export: right now, the main issue with "temperance" that I see is that it does not allow undoing that conversion on export. E.g. when you import with temperance the bones are rotated, and theoretically they could be flipped back on export, resulting in a perfect roundtrip with a (typical) conversion into the standards of the editing software.
Workflow Scenario
As a typical case for a workflow that is right now harder than it should be, these assets serve a good purpose:
https://cdn.jsdelivr.net/npm/@webxr-input-profiles/assets@1.0/dist/profiles/generic-hand/
If your goal is to import left.glb, do a few edits (e.g. paint the vertices) and export back, the current flows are:
-
import with default – "Temperance"
- bones will look "a bit weird" in Blender, but workable, roughly right direction
- upon export, file looks good
- when actually using that file, everything is broken, because WebXR assumes specific bone directions and now the model is broken for the original purpose. Worth noting that this is "silently breaking" – a user assuming that by default, importing and exporting will work, will now have a broken asset without clear cause.
-
import with "Blender"
- bones will look "very weird" as they're all pointing 90° in the wrong direction
- upon export, file looks good
- when using the file, it still works. Perfect roundtrip.
I also just came by this issue while working on some tooling which converts assets from a game to glTF for importing into Blender.
I spent a few hours debugging this thinking that my matrices were bugged or something, turns out that the default import setting completely obliterates the joint orientations from the imported file.
Here is an example of what it looks like when you import and export a glTF file with the Temperance setting:
ApplicationFrameHost_KxiQMDdW1L.mp4
I've always been a Maya user so I don't actually understand the reason why bones are represented the way they are in Blender, but to me it seems like there should definitely be at least some sort of warning notifying the user that the source bone orientations will get destroyed.
I also spent a few hours trying to understand why GLB files I was editing in Blender came out with a messed up rig that broke my downstream usage of those models.
Even if it's not made the default, I must agree with comments above that the naming of that option as "Blender (best for re-importing)" and even the expanded tooltip "Good for re-importing glTFs exported from Blender. " is somewhat confusing, and did not strike me as the right option to choose.
I was interpreting it as being concerned with making it better for later re-import to Blender. Whereas I just wanted something which allowed me to import and re-export a GLB without any modification of bone directions. i.e. something like "preserves bone directions".
@donmccurdy @hybridherbst Is there any way we can have blender write a extension to gltf so that down stream developers know that skeleton direction is not roundtripped.
This causes animations to be rebased onto a different skeleton than expected and requires difficult retargeting support in downstream applications.
Another thing I want to investigate is to use custom bone display to have bone "roundtripable", but displays as it is in temperance
I think it would be difficult to solve this with an extension. Comparing the asset.generator string of the two glTF models may get you just as far, e.g. checking whether both came from Blender.
If it's possible to preserve skeleton data by default while also having human-readable armatures within Blender by default, that would be a huge improvement.
Some thought about how we can improve it:
- Waiting this new display is available in Blender will help : https://twitter.com/sastuvel/status/1632413407528865798?s=20 (but probably not soon)
- Add Sphere custom bone shape to try to replicate the new drawing mode above
- Add Custom bone shape to have real bones roundtripable, but display as temperance.
For 2 last cases, we need to find a way to ignore the mesh used as custom shape at export.
More possibilities:
- Identifying that the armature was an imported one, and convert it back at export (including all stuff that will need to be converted back : IBM, animations, ... ). Quite complicated
Waiting this new display is available in Blender will help : https://twitter.com/sastuvel/status/1632413407528865798?s=20 (but probably not soon)
It looks good, so we discard the information about the Tail as length and treat it only as direction and draw a line connecting each Head, we can provide a Gizmo similar to most applications that utilize Node style bones.
This may be a wicked for those who utilize Tail, i.e., users who use Blender as their end environment such as making movie, not as a glTF editor, but only then should estimation be enabled as an option. And yeah, then, finally it should enable bone orientation estimation as optional.
But If it takes time to implement it, then for the time being, I hope to make the default value Blender as the provisional solution. (In any case, it should do so when that new Gizmo is implemented)
Also naming it Blender may not be good in the first place. If it is to be a choice value for the estimation, I suggest it be named something like None.
This will prevent the glTF importer from doing estimation by default, the same as the fbx and collada importers. Then, I don't have to explain every time to anyone who works with glTF character models in blender that the default for the glTF importer is to rotate the original bones madly.
I understand it's kind of a chicken-and-egg problem, but maybe the glTF importer changing the default to the proper option (not messing with the rotations → perfect roundtrips) is another incentive to get these new display modes into Blender? If there's a good-enough inbetween display option of course that's also great.
How is the progress on this? I still have to explain to new Blender users that when editing glTF, Khronos' Blender gltf-io is obvious different from the collada and fbx importers and discards the original bone orientation as the default.
@TokageItLab Good timing to ask!
Here is the PR that will land in 4.0 : #1986
I imported a model with arms and the marked bone has the wrong orientation, it should face to the right, i.e. be rotated 90 degrees. The bone must be oriented along a different axis. Import Settings: Bone Dir: Temperance, Guess Original Bind Pose: checked. I tried Bone Dir: Fortune and this bone is directed towards the upper phalanx.
Note that the same bone on the other hand is in the correct orientation. I imported other models that have the same skeleton and there this marked bone has a random orientation when importing, but the rest of the bones still have the correct orientation
Setting Bone Dir: Blender gives this result (second screenshot)
