Performance test models
javagl opened this issue · comments
This is related to other issues, but may warrant its own discussion.
- It is related to #56 , but this focussed on hardware- and API limitations
- It is related to KhronosGroup/glTF#1699 , but this seemed to bring up the question on the level of the specification, without the clear goal of having dedicated test models
It is also related to a recent discussion in #100 , which aims at having a model that challenges the runtime with many nodes. And I think that it is good to have such a model. But I think that it would be even better to not have a single model, but to have models that more or less systematically cover different dimensions along which the performance can be tested.
And there are many dimensions. Paraphrasing from a comment in the linked PR, there could be test models with ...
- 1000, 5000, 50000, and 100000 nodes
- 10000 (or more) nodes that refer to only 10, 100, or 1000 different meshes
- meshes that have 50, 100, 5000, 10000, 65635 vertices
- meshes that use 1, 10, or 1000 different textures
- textures of size 32x32 ... 4096x4096
All these can be combined, yielding a many-dimensional space with billions of potential test models.
The question then is:
What should be tested?
I think that people with experience in engine/viewer development might have some ideas what the pain points (i.e. the interesting points) could be. And with a clearer idea about these interesting points, one could set up a collection of test models. (Not necessarily as actual assets - preferably, with a mechansim for generating them on demand).
As one overly specific example:
The following is a set of assets that each contain 4096 nodes, arranged in a grid. The only difference between these assets is the number of meshes. The number of meshes is [1, 8, 64, 512, 4096], meaning that in the first case, all nodes refer to the same mesh, and in the last case, each node refers to a different mesh:
output-numMeshes-2024-02-18.zip
The meshes are still equal: They have no textures, no materials, and each has only one mesh primitive for a plane with 10x10 points, and all these mesh primitives share the same accessor data (otherwise, all this wouldn't fit into 130KB)
An aside: The visual appearance is boring...
And even though "looking cool" is not the goal of this sort of model, that doesn't mean that such models can not look cool. For example, the same model with 4096 nodes and 4096 meshes, but this time with 32 different materials each with a different texture would look like this:
There are still many dimensions fixed here. For example, all textures have the size 1024x1024, and all meshes still have a single primitive plane with 10x10 points. Arbitrary combinations of that would look similar, but could pose other challenges for engines.
A trippy outtake, to open up the space further: These are 4 viewers, showing a model with ~10000 nodes that are all animated with a translation- and rotation component:
These may be varied along the number of nodes, the animated properties, the number of interpolation steps, the interpolation types, and much more...
(Only https://gltf-viewer.donmccurdy.com/ is somewhat smooth, but of course, that's not really a fair (or even just sensible) comparison for now)
In view of #100 : Here are ~10000 asteroids (100 meshes + textures), each one rotating and revolving around the center at different speeds:
(Still, none of them are colliding 😎 )