generic model from Mixamo not working
grimdeathr opened this issue · comments
This model does not work.
Its the standard ybot from mixamo that i imported into Blender and re-exported as a GLTF
Is there any special process to follow?
Im testing the 006_ik_retarget with babylonjs if that helps
same thing with the threejs examples
What exactly isn't working? What are you trying to do? Do you have a repo that I can see what your doing?
In general, FBX import > Gltf Export can be dicey sometimes. If you're including animations, the export processes can mess with the bind pose values of the node which you'd need specific settings to get things out correctly. Fbx import also adds rotation & scale to the root elements which tends to get in the way sometimes. I typically make sure to apply all transformations from the root down to all the child elements to make sure there isn't any node transforms lurking around but doing this will trash any animations your trying to export, so I only do that for meshes only. If animations is the main thing to export, especially from mixamo, there are extra steps I take using the node transforms & re compute the bones on the pose object since you shouldn't change them on the armature itself.
All the animations are from mixamo, I purposely export them without the mesh data & only use the skeleton. You can try the following link and see if you can at least get the skeleton to be visible. From there, check to see of the meshes your loading have the node transform applied. The skeleton is using it to get a proper setup for IK in webgl, so if the mesh itself has the same transform applied to tit then its doubling up on the rotation & scale, it might even scale it to the point its not visible anymore.
https://github.com/sketchpunklabs/ossos/blob/main/examples/threejs/006_ik_retarget.html#L209
My sample is the zip i posted with the animation which does not show the mesh, it looks like the bones are there ? but not seeing the skinned mesh.
Blender is what i use to convert the models to gltf2, should I be using something else?
I am testing the mesh out on your animations and thats where the issue is.
Its good to know you are using animations with no mesh data, ill keep that in mind for future.
https://www.donmccurdy.com/2017/11/06/creating-animated-gltf-characters-with-mixamo-and-blender/
I'll see if this helps in any way
--Edit
Didn't help
importing and exporting the Tina model for example in blender 3.1 causes the arms to flip, thats one difference ive noticed.
Did you check to see if there is a transform applied to the 3d Mesh? Its position should be 0,0,0, quaternion 0,0,0,1 and scale 1,1,1.
Did you try to see if just the skeleton renders fine using the link i sent which handles how to load in the armature of mixamo's ybot?
The gltf seems to be fine when I view it on a GlTF viewer, just huge. To make it easy for me to debug an issue, a temp repo with a sample of what your doing that I can clone & run npm on would speed things along.
importing and exporting the Tina model for example in blender 3.1 causes the arms to flip, thats one difference ive noticed.
Yea... GLTF importing is kinda bad in blender, i usually opt for fbx since it gives me the least amount of issues to work through.
Did you try to see if just the skeleton renders fine using the link i sent which handles how to load in the armature of mixamo's ybot?
Looks like this, cant see the mesh but am i right to think youre not supposed to see the mesh with this one?
// new NabbaRig().load( {boneView:true, mesh:true} ).then( rig=>{
// rig.mesh.position.x = 1;
// rig.boneView.position.x = 2;
// Ref.rigs.push( rig );
// }),
cant see the mesh with this one either ^
console.log( mesh.position, mesh.quaternion, mesh.scale ) ?
Where should i call that ?
here ?
new NabbaRig().load( {boneView:true, mesh:true} ).then( rig=>{
console.log( rig.mesh.position, rig.mesh.quaternion, rig.mesh.scale )
rig.mesh.position.x = 1;
rig.boneView.position.x = 2;
Ref.rigs.push( rig );
}),
gives this result :
missing quaternion and scale
logging the name shows its the wrong mesh ?
Second one has the alpha_joints object removed and the result is the same
My code, fairly standard stuff just a t-pose
When i change the MixamoIKAnimatorRig model to the Rumba Animation, it does almost the same thing
except i can see the bones are actually moving with bone view
but again the mesh does nothing, even though ive tested the mesh bones in blender ,they do affect the surface.
post what you have to a repo. Its to hard to debug this way.
thats the thing, i havent changed much, the only thing im doing is using the ybot zip in examples folders and just calling the zip
https://github.com/grimdeathr/res
here is the repo
https://github.com/grimdeathr/ossos
the code, however little changes are there are now in that repo
cools. which file are you working on so I know where to look.
The babylonjs 006_ik_retarget.html is one and threejs is the other one
for the models im just using the ybot/yb.gltf folder in models in the _res repo
uploaded a random fbx to mixamo (luigi model) and converted to gltf and tested it out, same exact issue
I tested out every model and the ones in red do not load at all using the ik_retarget.html
Something isn't right about the gltf export of your ybot. I remove the transform from the skeleton and just render the mesh as is without skinning. The skeleton is HUGE and laying on the floor. The Model itself is the size you expect and right side up, double checked its transform and it doesn't have any so this is what the verts actually look like. I even have the bounding area being drawn to just make sure this is really the size of the mesh. Mesh & skeleton are WAAAY off from each other, no wonder things dont work. Technically the model should be just as big and laying on the floor, that's how its always been for me when I exported FBXs to GLTF on blender 2.8. I dont know if there are issues with blender 3.
The only thing I can do is install blender 3.1 & you upload the original FBX file to the repo for me to grab and I can play with it later tonight or some point this week to see whats up with the exporter, if there is specific settings that needs to be set to get things lined up.
Ok thanks I think the one last point was about various models in your own repo working and not working, are these not also related?
I tested out every model and the ones in red do not load at all using the ik_retarget.html
this
so this is what the verts actually look like. I even have the bounding area being drawn to just make sure this is really the size of the mesh. Mesh
I will look at using 3ds max export
https://github.com/sketchpunklabs/ossos/blob/main/examples/babylonjs/_000_debug_mesh.html
I put together a page to make it easy to debug the mesh and skeleton to see whats really in the gltf. If you want to play around on your own to figure the right settings to export the gltf file, this should help you test the results.
You want to be able to see the skeleton and mesh match up in scale & rotation
Not like this with it being mismatched for some reason.
To me it look like the transform ( rotation, scale ) thats on Armature node in blender is being applied to the mesh but not the skeleton on export.
I tested out every model and the ones in red do not load at all using the ik_retarget.html
Thats because in your repo I saw you trying to squeeze YBot into Nabba's Rig Object. Each model needs its own rig object tailored for ot. Basic characters can be auto rigged as you saw some was easy to load in, but most can't because of how the mesh or skeleton is setup.
If you look in the demo folder you'll see a rig folder to see further examples of how ik rigs are setup for various different types of characters.
https://github.com/grimdeathr/ossos/tree/main/examples/_demo/rigs
I tested out every model and the ones in red do not load at all using the ik_retarget.html
Thats because in your repo I saw you trying to squeeze YBot into Nabba's Rig Object. Each model needs its own rig object tailored for ot. Basic characters can be auto rigged as you saw some was easy to load in, but most can't because of how the mesh or skeleton is setup.
If you look in the demo folder you'll see a rig folder to see further examples of how ik rigs are setup for various different types of characters. https://github.com/grimdeathr/ossos/tree/main/examples/_demo/rigs
I see your point about the NabbaRig and I was trying to essentially write the least amount of code to get it working which is to say
class NabbaRig extends CharacterRig{
constructor(){ super(); }
async loadAsync( config=null ){
const gltf = await Gltf2.fetch( '../_res/models/nabba/nabba.gltf' );
this._parseArm( gltf, false ) // Create Armature
._autoRig() // Auto BipedRig
if( config?.mesh != false ) this._skinnedMesh( gltf, 'cyan', config );
if( config?.boneView ) this._boneView( config );
return this;
}
}
i thought at a minimum i would have to do these things, but it might be that it requires further work, then in that case it would help to do a generic rig - ill see if i can write a new rig for the ybot or really any model
Would the CharacterRig work?
As starting point you can use with nabba if you know you have a simple model or tina if its a bit more complex with hair or clothes. YBot has 2 meshes and 1 skeleton. The util object will only grab the first mesh & skeleton, you'll need your custom rig to handle getting & using two models with one skeleton.
Either way, you need to get the export right first, without the bones & mesh lined up nothing is going to work.
thanks for all your help
cant get it to work unfortunately
post the FBX that you get from mixamo onto the test repo you setup. I'll grab the file when I get time, install blender 3.1 & see what I can do to get the file exported correctly.
yb.zip
This zip is a download from Mixamo and has 2 FBX files, includes 2 default generic characters : ybot and xbot
I've tested on blender 2.8 to 3.1 and they all have this issue..
No progress I guess?
I have blender 3.1 installed, I can't load the two fbx files you sent. Gives me a weird message about the FBX version is to old.
I just downloaded a free model from this website. Have a look on here there are many models. The issue in all of this is not being able to use either a rig from blender or a rig from Mixamo.