More edge cases for interleaved buffer views
donmccurdy opened this issue · comments
This model was reported as not working in three.js, and it doesn't appear to work in the Sample Viewer either. The buffer layout has a single mesh primitive with several vertex attributes, which are interleaved, but each attribute accessor has its own buffer view — rather than setting byteOffset on each accessor, a byteOffset is assigned to the accessor's unique buffer view.
This passes the validator, and seems valid according to the spec too. It's not a data layout I'd want to encourage, but the bug it identified in three.js could technically happen in some reasonable cases too, so maybe it's worth including a test case here.
It appears to work in BabylonJS already, although the accessor three.js failed on was (by chance) an unused UV set, so if BabylonJS ignores unused UVs it might have dodged the issue entirely that way.
It appears to work in BabylonJS already, although the accessor three.js failed on was (by chance) an unused UV set, so if BabylonJS ignores unused UVs it might have dodged the issue entirely that way.
Babylon isn't ignoring unused UVs as users may add textures after loading. It seems the math works out fine in Babylon so I think it's a bug in three.js and the sample viewer? But even if the math turns out to be bad for typed arrays, the Babylon glTF loader allows invalid typed array offset/length combinations due to backwards compatibility since at some point it worked out of the box.
It's not a data layout I'd want to encourage, but the bug it identified in three.js could technically happen in some reasonable cases too, so maybe it's worth including a test case here.
We avoid putting edge cases that ideally should not exist. Otherwise, the matrix of cases get unwieldly rapidly. I'm not sure there is a reasonable case for this one. Maybe we can add as a negative test case, but it will blur the definition a bit. If we come up with a reasonable case, I'm all for it.
The data layout is unusual, at least, and I can imagine that some tools choke on that.
(One of my utilities reported an error: "The accessorModel has an offset of 0 and 5 elements with a byte stride of 44, requiring 220 bytes, but the buffer view has only 208 bytes" - which sounds like a reasonable error message, but I assume that I just messed up the error check here. Or conversely: It seems very unlikely that such an error wasn't covered by the validator if it was really invalid...)
Looking at the JSON, this refers to this accessor
{
"bufferView":2,
"componentType":5126,
"count":5,
"max":[
1.0,
1.0
],
"min":[
0.0,
0.0
],
"type":"VEC2"
},
and this buffer view
{
"buffer":0,
"byteOffset":60,
"byteLength":208,
"byteStride":44,
"target":34962
},
It might still be valid, considering that the fifth element in this case is only a VEC2 of float, and thus, does not "occupy" any of the bytes that one might expect to be present. (I still wonder where the stride of 44 comes from, but will have to do further checks against the spec and things like KhronosGroup/glTF#1198 to be sure here).
The byteLength isn't really constrained in the spec. If it is allowed to be smaller than "numElements * byteStride", this could probably be made clearer at some point.
The full JSON from the model, maybe someone wants to have a look...
bloc.gltf
{
"asset":{
"generator":"Guntha's Engine MeshBuilder::saveToGLB",
"version":"2.0"
},
"scene":0,
"scenes":[
{
"name":"Scene",
"nodes":[
0
]
}
],
"nodes":[
{
"mesh":0,
"name":"Mesh"
}
],
"meshes":[
{
"name":"Mesh",
"primitives":[
{
"indices":0,
"attributes":{
"POSITION":1,
"TEXCOORD_0":2,
"COLOR_0":3,
"NORMAL":4
}
}
]
}
],
"accessors":[
{
"bufferView":0,
"componentType":5125,
"count":12,
"max":[
4
],
"min":[
0
],
"type":"SCALAR"
},
{
"bufferView":1,
"componentType":5126,
"count":5,
"max":[
1.0,
1.0,
2.0
],
"min":[
-1.0,
-1.0,
-0.0
],
"type":"VEC3"
},
{
"bufferView":2,
"componentType":5126,
"count":5,
"max":[
1.0,
1.0
],
"min":[
0.0,
0.0
],
"type":"VEC2"
},
{
"bufferView":3,
"componentType":5126,
"count":5,
"max":[
1.0,
1.0,
1.0
],
"min":[
0.0,
0.0,
0.0
],
"type":"VEC3"
},
{
"bufferView":4,
"componentType":5126,
"count":5,
"max":[
0.5773503184318543,
0.5773503184318543,
1.0
],
"min":[
-0.5773503184318543,
-0.5773503184318543,
0.5773503184318543
],
"type":"VEC3"
}
],
"bufferViews":[
{
"buffer":0,
"byteOffset":0,
"byteLength":48,
"target":34963
},
{
"buffer":0,
"byteOffset":48,
"byteLength":220,
"byteStride":44,
"target":34962
},
{
"buffer":0,
"byteOffset":60,
"byteLength":208,
"byteStride":44,
"target":34962
},
{
"buffer":0,
"byteOffset":68,
"byteLength":200,
"byteStride":44,
"target":34962
},
{
"buffer":0,
"byteOffset":80,
"byteLength":188,
"byteStride":44,
"target":34962
}
],
"buffers":[
{
"byteLength":268
}
]
}
If it is allowed to be smaller than "numElements * byteStride", this could probably be made clearer at some point.
That was exactly my mistake, and probably the mistake of glTF-Sample-Viewer too. 😇 The spec actually does spell this out, I just wasn't looking for it;
Each accessor must fit its bufferView, i.e., accessor.byteOffset + STRIDE * (accessor.count - 1) + SIZE_OF_ELEMENT
must be less than or equal to bufferView.length.
Visually, here's the problem. Suppose byteStride were 8 bytes, with two attributes, and 4 vertices. Each vertex is 4 bytes, let's say that's 3 bytes + 1 byte of padding.
xxxx|----|xxxx|----|xxxx|----|xxxx|----|
----|yyyy|----|yyyy|----|yyyy|----|yyyy|????
My implementation assumed it would encounter the first attribute (X) in the buffer first, and it constructs a TypedArray then. When we encounter attribute X first, that's fine — the length of the buffer is exactly count * byteStride. But if we encounter Y first, we'll overshoot the buffer by 4 bytes. This model hit the edge case because it assigned each attribute to a unique bufferView, and just really broke my assumption for how the data should be packed. Technically it's valid, but the author agreed it was a mistake.
However, I'd be willing to bet my implementation would also break in a much more reasonable situation, where all attributes are in the same bufferView, but they're listed under primitive.attributes in reverse order of the buffer layout. For example:
{POSITION: 0, NORMAL: 1} → {NORMAL: 1, POSITION: 0}
By luck, that's not something exporters are very likely to do, but I do think it might be a reasonable enough situation to test for here.
(Not sure how I overlooked that part of the spec - I did several CTRL+F passes with the related keywords...)
Here's an image of the actual data layout of the asset. Even if this case is not dedicatedly handled by the asset generator, maybe the model could be added as one of the "simpler test models" in the glTF Sample Models repo (along with the "interleaved cube"), so that importers are aware of that case.
It's not a good data layout — it'd very likely cause the engine to lose the benefits of interleaving — so I wouldn't want to put it in glTF-Sample-Models personally. If we find a "good" model that hits the same issue then we should include it either there or here.
However, I'd be willing to bet my implementation would also break in a much more reasonable situation, where all attributes are in the same bufferView, but they're listed under primitive.attributes in reverse order of the buffer layout.
I thought this would break, but it didn't — both three.js and the sample viewer do handle this correctly. So I guess they're just broken in the more pathological case of the original model.
I'm not sure whether any action is necessary, in that case. I understand @bghgary's preference to not host edge cases that shouldn't exist... is this enough of an edge case that we should put an implementation note or even a bugfix in the spec?
/cc @lexaknyazev
I'd have to take some time to wrap my heads about the details, but if I understood that correctly, then the main issue is (very roughly speaking) that a straightforward implementation of a viewer would upload the 4 buffer views to the GPU, even though they contain the same data, causing some redundant redundancies. (The viewer could figure that out, but it would be complicated, so no viewer would do that by default)
I wonder whether we could find a way to more precisely say what the term "good data layout" means. A quick search on the spec shows that it does not contain the word overlap (as in "buffer views may/may not/should not overlap"), but maybe it's still there, phrased differently. (And I think that overlap is a core point here...)