[rmodels] Incorrect animation of glTF model
paulmelis opened this issue · comments
Please, before submitting a new issue verify and check:
- I tested it on latest raylib version from master branch
- I checked there is no similar issue already reported
- I checked the documentation on the wiki
- My code has no errors or misuse of raylib
By the way, the FAQ currently reads:
GLTF: This file format supports animation but raylib is not able to load them at this moment. GLTF animations loading was implemented in the past but GLTF is a quite complex file format and could implement animations in many ways, raylib used to fail a lot on loading them and support was removed.
But there is LoadModelAnimations()
which supports GLTF? So seems the FAQ entry is out of date?
Issue description
While trying to figure out how to make #4037 work well together with animated models, I noticed current support for animations isn't entirely bug-free (hence making the glTF transform work on top of it somewhat challenging).
For example, with https://github.com/KhronosGroup/glTF-Sample-Models/blob/main/2.0/BrainStem/glTF-Binary/BrainStem.glb (which needs #4053 to handle the missing bone names) the initial load of the model, without applying UpdateModelAnimation()
is okay (which is not surprising, as there's only a single mesh):
But when applying UpdateModelAnimation()
in the render loop, the transforms based on the armature make parts fly all over the place incorrectly:
But then a very similar model like https://github.com/KhronosGroup/glTF-Sample-Models/blob/main/2.0/CesiumMan/glTF-Binary/CesiumMan.glb does animate correctly, and I don't immediately see the big difference between the two files causing the issues.
Environment
Arch Linux, GTX TITAN
Code Example
#include <string>
#define SUPPORT_FILEFORMAT_JPG
#include "raylib.h"
#define PLATFORM_DESKTOP
#if defined(PLATFORM_DESKTOP)
#define GLSL_VERSION 330
#else // PLATFORM_ANDROID, PLATFORM_WEB
#define GLSL_VERSION 120
#endif
int main(int argc, char *argv[])
{
std::string modelfile("../models/robot.glb");
if (--argc == 1)
modelfile = argv[1];
const int screenWidth = 950;
const int screenHeight = 540;
SetConfigFlags(FLAG_MSAA_4X_HINT);
InitWindow(screenWidth, screenHeight, "play animation");
Model model = LoadModel(modelfile.c_str());
BoundingBox bbox = GetModelBoundingBox(model);
printf("Model bbox min: %.6f, %.6f, %.6f\n", bbox.min.x, bbox.min.y, bbox.min.z);
printf("Model bbox max: %.6f, %.6f, %.6f\n", bbox.max.x, bbox.max.y, bbox.max.z);
int animCount = 0;
ModelAnimation* modelAnimations = LoadModelAnimations(modelfile.c_str(), &animCount);
printf("Loaded %d animation(s)\n", animCount);
Camera3D cam = (Camera3D){ 0 };
//cam.position = (Vector3) { bbox.max.x*2.5f, bbox.max.y*2.5f, bbox.max.z*2.5f };
//cam.target = (Vector3) { 0.5f*(bbox.min.x+bbox.max.x), 0.5f*(bbox.min.y+bbox.max.y), 0.5f*(bbox.min.z+bbox.max.z) };
cam.position = (Vector3){ 3.0f, 3.0f, 3.0f };
cam.projection = CAMERA_PERSPECTIVE;
cam.up = (Vector3){ 0.0f, 1.0f, 0.0f };
cam.fovy = 45.0f;
bool animating = false;
SetTargetFPS(60);
int fc = 0;
// Main game loop
while (!WindowShouldClose())
{
float dt = GetFrameTime();
UpdateCamera(&cam, CAMERA_ORBITAL);
if (modelAnimations && animating)
{
fc++;
fc %= (modelAnimations[0].frameCount);
UpdateModelAnimation(model, modelAnimations[0], fc);
}
BeginDrawing();
ClearBackground(RAYWHITE);
BeginMode3D(cam);
DrawModelEx(model, (Vector3) { 0.0f, 0.5f, 0.0f }, (Vector3) { 0.0f, 1.0f, 0.0f }, 0.0f, (Vector3) { 1.0f, 1.0f, 1.0f }, WHITE);
DrawGrid(10, 1.0f);
EndMode3D();
EndDrawing();
if (IsKeyPressed(KEY_S)) TakeScreenshot("animation.png");
if (IsKeyPressed(KEY_A)) animating = !animating;
}
UnloadModel(model);
CloseWindow();
return 0;
}
Edit: first linked glb was wrong, should be BrainStem.glb, now fixed
@paulmelis Thanks for reporting about the FAQ, just reviewed it.
About the different models animations working or not, I don't know the reason but I can guess is related to the bones hierarchy and its transformations. Is it possible to export a GLTF file with the transformations "baked" by bone?
Were these models exported by blockbench?
Were these models exported by blockbench?
I don't know. They're part of the glTF sample repo, but no idea how they were created.
Hmm, one difference I see that might be of influence is that CesiumMan.glb only has one mesh with one primitive, while BrainStem.glb also only has one mesh but that mesh has a few dozen primitives (each with their own JOINT_0 and WEIGHTS_0 arrays). And primitives are converted into separate raylib Mesh's during loading.
Were these models exported by blockbench?
Hello, I tested exported models by blockbench of the cobblemon's pokemon. In general, feet and body are placed correctly during animation, but the head and other bones like leaves on a ivysaur go right in the center. I have faced a similar issue with the Fox of the glTF sample repo.
@paulmelis @Arcod7 thanks for the further investigation of this issue! It seems issue could be related how meshes are exported by blockbench
but also how they are loaded by raylib...
Here's an illustrated example : The Torterra model in blockbench exported to glb works great in blender, but breaks in raylib (on the latest github dev version).
Here's a short clip
In Blockbench, we can see that the parts breaking are the ones indented more than 1 in the body. For example the head is affected (body->torso->head) and the feet too while the leg is well placed (legs are not part of torso).
During the animation, I noticed that while the head and others are placed in the center of the model, the feet seems to be at the center of the leg and not the body.
I'm highly motivated to try to solve the bug, do you have any idea that could help me ?
@Arcod7 Mmmh... interesting, it seems that node transformations are only considered for the first hierarchy level but not applied down to the other hierarchy levels.
Okay, do you think you will have time for this soon ? If there's something else I can provide to help, you can tell me !
@Arcod7 I'm afraid I don't have time to review it. I neither implemented current loader (only vertex data loading) so it would take me a lot of time that unfortunately I don't have at the moment.
@VitoTringolo Wow! It looks great!!!
Can you maybe try this model to?
Player-Female.zip
Hello @VitoTringolo, you did great, it looks stunning !
I have a project with raylib that ends in 2 days and I would love to try out your fix, could you push it on your fork ?
Also did you only touched the translation part or you also fixed rotations ?
On the model provided by @MrScautHD (and a lot of my models), parts of the body disappear with the rotation applied.
For exemple, the head of the Player-Female model is only present at the frame first and last frame of the run animation. It's the same thing with the arm of the player with the idle animation.
Run animation: (only scale and rotation applied, I disabled transformations)
Idle animation: (only scale and rotation applied, I disabled transformations)
@MrScautHD @Arcod7 is this model created with Blockbench
?
@MrScautHD @Arcod7 is this model created with
Blockbench
?
yea
@MrScautHD @Arcod7 is this model created with
Blockbench
?
My models are also coming from blockbench (bbmodel exported to glb). They are working great with blender (not a single issue).
Can you maybe try this model to? Player-Female.zip
Sure thing!
This is the result that seems to be correct!
If you have other models to test, please share :)
Hello @VitoTringolo, you did great, it looks stunning ! I have a project with raylib that ends in 2 days and I would love to try out your fix, could you push it on your fork ?
Also did you only touched the translation part or you also fixed rotations ?
On the model provided by @MrScautHD (and a lot of my models), parts of the body disappear with the rotation applied. For exemple, the head of the Player-Female model is only present at the frame first and last frame of the run animation. It's the same thing with the arm of the player with the idle animation.
Run animation: (only scale and rotation applied, I disabled transformations)
![]()
Idle animation: (only scale and rotation applied, I disabled transformations)
![]()
Before we were completely not considering the transform matrix when the animation channel was NULL! This was the main issue! So to answer your question, yes I am now considering the translation the rotation and also the scaling factor for each joint
It looks so nice 😊 !
Can you try the second and third animations of this model ? This is where I was getting issues with head and arms which disappear
My models are those from cobblemon
These are .bbmodel, it can be exported to .glb by blockbench (make sure to check the "Export Groups as Armature" when exporting)
If you want a sample of exported models I can provide it.
This fixes almost everything on the cobblemon models, thanks a lot !
There is juste one last thing, sometimes a bbmodel animation has an animation length of 0 which means that there's no end to the animation but it still playable.
Do you have an idea of what's going on with raylib ?
This fixes almost everything on the cobblemon models, thanks a lot ! There is juste one last thing, sometimes a bbmodel animation has an animation length of 0 which means that there's no end to the animation but it still playable. Do you have an idea of what's going on with raylib ?
I'm not sure I understand what you described, what do you mean with "animation has an animation length of 0 "? Can you share a model with these properties?
Sure ! Here's a model with a ground_walk working at index 1 (animation length is set to 2 in blockbench) and a ground_idle that's not working as expected at index 0 (animation length is set to 0 in blockbench)
torterra.glb.zip
https://github.com/raysan5/raylib/assets/77203393/cb7a6135-09f9-4f23-98ae-6356668612f9
I link a video to illustrate the issue
Have a good day !
In my opinion the solution for this is to set a value different then zero before exporting the model from Blockbench.
I think that is the right solution. Otherwise, it should be checked if the glTF official specs allow other behaviours.
@MrScautHD What is the license of the Player-Female.glb
model? Is it free?
@MrScautHD What is the license of the
Player-Female.glb
model? Is it free?
Its mine, so feel free to use it. 😊
@VitoTringolo and @raysan5 What about this error?
It really spamms the console like 1000 lines and the model still work.
@VitoTringolo and @raysan5 What about this error? It really spamms the console like 1000 lines and the model still work.
Oh yeah! That's because of this line of code:
if (FloatEquals(tend, tstart)) return false;
I think I can change to return true, without any problem to avoid flooding!