3D Tiles Roadmap

[Patrick Cozzi]

New to 3D Tiles? See Introducing 3D Tiles.

This work is in the 3d-tiles branch.

In pseudo-priority order within each section:

Draft 1.0

• Cesium 3D Tiles implementation
  • Inspector, #4846
  • Is the load order correct? The unit tests look OK; however, for example, the buildings on the right should load sooner: http://cesiumjs.org/WashingtonDC/
  • Memory issues?
    • Unloading issue, #4673
    • Empty tile memory usage, #3453
    • Crash with high memory usage, #4549
  • glTF 2.0, #4009
  • Merge request scheduler into master, #3476
    • IE performance, #4166
    • Unable to have multiple CesiumWidets on the same page in 3d-tiles branch #5226
  • Loose OBBs #4387
  • Translucency and artifacts
    • Combine translucent back/front commands into a single translucent command, however this will look worse when OIT is not supported.
    • Pick translucent features, see TODO in Cesium3DTileBatchTableResources.getVertexShaderCallback
    • Finish derived commands, especially Cesium3DTileBatchTableResources.getAddCommand, #3806
    • Frustum artifacts #4381
  • Finish Styling GLSL-backend CesiumGS/3d-tiles#140
  • Styling - vector support for built-in functions #4865
• Doc and Examples
  • cesium.com hosted examples in the Sandcastle example
  • Tutorials
    • Using the Cesium API to load 3D Tiles and interact with individual models.
    • Declarative styling.
  • Copy 3D Tiles Sandcastle tests from the Specs/Data folder to the Apps/SampleData folder.
    • Or store test data in cesium.com and download on npm install or something similar?
• Any outstanding TODOs in the code
• Spec draft 1.0 issues
• Tools - 3d-tiles-tools
• drillPick point cloud - #4895

Post Draft 1.0

• Vector tile - instance geometry volumes, #4665
• Vector tile - z-order, #4665
• Vector tile and geometry tile - clipping planes
• If requested - classify with i3dm, #6033
• Technique like #5128 for fuse tilesets/terrain at runtime
• Draco compression (then quantize at runtime), #5120
• Vector tile format. CesiumGS/3d-tiles#25, #2172
  • Increase bounding volume by pixel size like BillboardCollection and PointCollection. See the Cesium3DTile constructor.
• Point cloud attenuation example using declarative styling (and vertex shader culling?)
• scene.pickPosition returns incorrect position #4368
• Better error metric than SSE? #4043
• Reuse Primitive Batch Table, #4328
• Add convenience functions to the Cesium API for querying features based on the batch table hierarchy - like getFeaturesByProperty(propertyName, value) and getFeaturesByClass(className)
• Cache Management, @lasalvavida
  • Refactor TileReplacementQueue to use DoublyLinkedList?
  • Example code for app that uses tileUnload and tileVisible events to re-style a tile that was unloaded, e.g., by tracking features that changed or higher-level context like how 3D Tiles styling works.

Later + Ongoing Performance

• 3D Tiles spec and Cesium implementation
  • Add credit to tileset.json, CesiumGS/3d-tiles#55
  • Download multiple children in a single request, CesiumGS/3d-tiles#9
  • More declarative styling, CesiumGS/3d-tiles#2
  • Custom terrain tile format (instead of b3dm). Initially support overlaying separate imagery layers?
  • Refinement parameters. CesiumGS/3d-tiles#15
  • Texture LOD for b3dm/i3dm
  • Cleanup: GLSL bit extract helpers, #4315 (comment)
• Cesium 3D Tiles implementation
  • AO for 3D buildings (integrate gltf-pipeline)
  • tileNewlyVisible and tileNewlyNotVisible events
  • Performance: Best SSE in center
  • Performance: Add extra frustum planes to account for how wide the FOV is and that the viewer might not notice objects on the edge and in the distance (horizon views for sure, maybe not aerial), e.g, two planes with normals perpendicular to the view direction
    • Also, tell users to use a more narrow FOV
  • Performance: cull/LOD select with union of children bounding volumes instead of the tile's BV. Does the extra CPU overhead result in fewer tiles on average? Is it a performance win?, #2861
  • Performance: for fast moving views (like flying from home view to Canary Wharf), increase the geometric error so few tiles are requested and then only shown briefly or are already out of view. Could track the viewer's velocity and perhaps heuristics for if a requested tile was immediately culled
  • Performance: optimize replacement refinement - instead of requesting all children when a tile needs to refine, request only the visible ones, and then allow refining a tile if all the required visible tiles are loaded (instead of all - including invisible - tiles). When the camera moves, and children that are not yet loaded are needed, either render the parent or render the available children and the parent with clipping, #4287
    • Still need to clip the parent to the loaded children
  • Performance: skip levels when refining with replacement refinement
    • For additive replacement, load tiles deep in the tree first, then ancestors
  • Performance: optimize batch table memory usage, see PERFORMANCE_IDEA in Batched3DModel3DTileContent.js and Cesium3DTileBatchTableResources.js.
  • Evaluate LZ4m: A Fast Compression Algorithm for In-Memory Data
  • Performance: Turn off opaque textures for picking pass, e.g., in Miami for rollover picking.
  • Performance: exploit temporal coherence in sortChildrenByDistanceToCamera
    • Sort by pixel size instead of distance? What about large bounding volumes with sparse contents?
    • Is selectTiles better implemented with a priority queue instead of explicit sort?
  • Performance: nest spatial data structure to optimize additive refinement, CesiumGS/3d-tiles#11
  • Performance idea: prefetch children or siblings? For example, see Figure 3. Track stats over time to change algorithm or disable.
  • Performance: Reduce per-DrawCommand overhead
  • Performance: Fuse nodes at runtime to reduce the number of DrawCommands.
  • Performance: "to avoid jarring popping artifacts, it could be useful for a runtime to scale the viewerRequestVolume to be slightly larger, and then check the geometricError for prefetching (but not rendering until the viewer enters the unscaled volume).", CesiumGS/3d-tiles#101
  • Performance: devicePixelRatio on mobile, #3288
  • Performance: Omni-directional Relief Impostors (ORI), #4908 and https://groups.google.com/forum/#!topic/cesium-dev/tCCooBxpZFU%5B101-125%5D
  • Point clouds: hidden point removal, anisotropic filling, and shape depiction, and compute normals for lighting. Also edge-preserving blur in Game Engine Gems 3 and CIS 565.
  • Labels on buildings
  • Occlusion culling. Drive refinement with VMSSE?
  • Morph between LODs
  • Support 2D and Columbus view. #4884
  • Make PointGeometry and PointAppearance public. #3203
  • Job scheduler, #2655
  • Memory-efficient (bitfield) per-building cast/receive shadows if needed
  • More Dynamic SSE #4307
    • Increase SSE as the view gets more horizon
    • Load tiles up to a max distance first, then load tiles in the distant if the camera doesn't move
    • Fog to help visual quality?
  • Cache Management
    • When skip LOD is used, remove non-leaf tiles from the cache if they are touched by not rendered
    • Research strategies for proactive cache trimming, e.g., number of seconds/frame a tile was not selected, how far out of view a tile is, etc.
      • Potentially track statistics to see if the cache is trashing, and then adjust the replacement algorithm.
    • Is it possible to come up with a better default for maximumNumberOfLoadedTiles since the use cases are so diverse?
    • Unload sub-trees from tiles with tileset.json content when all their content is unloaded
    • Vertex/texture cache across tiles
    • Provide more precise cache size than number of tiles? Count composite tiles as more? Include geometry/texture size with each tile? Perhaps number of tiles is the easiest thing to understand.
  • POP buffer for progressive replacement refinement
• Review "Interactive inspection of complex multi-object industrial assemblies", #4908

Older

• gzip tiles? CesiumGS/3d-tiles#61, @pjcozzi
  • 404 for missing compressed tile, #3508, @pjcozzi
• Finish/optimize i3dm tile format. CesiumGS/3d-tiles#33. @lasalvavida
• Finish point clouds tile format. CesiumGS/3d-tiles#22.
  • Compression notes: quantization; Morton curve to minimize deltas?; compress color to 11-bits like in Page 3? @lasalvavida
  • Rendering notes: point size attenuation, including vertex shader culling; styling, #3639
• Evaluate potential Batch Table changes, CesiumGS/3d-tiles#32, @pjcozzi
• Instanced models are not being cached correctly between different ModelInstanceCollections, #4144
• 3D Tiles cache not cleaning up fully #4147
• tiles invisible and cut, #4333
• Do not set Model.incrementallyLoadTextures to true (default)
• b3dm with external textures reference error #3892
• Fix test failures when running with Karma (makes Travis CI fail): npm run test -- --exclude WebGL --browsers Electron --failTaskOnError --suppressPassed
• Implement WEB3D_quantized_attributes. @lasalvavida
• Remove workaround2657 in Cesium3DTile.js, #2657. @lasalvavida
• Fix replacement refinement when external tileset is loaded, #3517, @lilleyse
• Reorganize load/replacement (like terrain) to support shadows, @bagnell
• Add fog, @bagnell
• TileOrientedBoundingBox debug volume bug, #4581
• debug show view frustum, #3856 (comment), @lasalvavida
• Fix mouse in Sandcastle example, #3614
• Expose bounding volume, #3138
• Support replacement color in addition to highlight color? See Cesium3DTileBatchTableResources.getFragmentShaderCallback.

[e-andersson]

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling? We're quite interested in tileset loading/unloading since we'll potentially have a large number of tilesets available to the viewer, causing a lot of initial tiles.json loading and repeating iteration over primitives.

[Patrick Cozzi]

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling?

It means lower-level tile handling, but thanks for reminding me about which Cesium3DTileset to load. I added that to the roadmap. :)

If you are interested in working on either, I'm happy to discuss in more detail.

[Patrick Cozzi]

Stick with the Cesium3DTile prefix for names?

After doing a bunch of reference doc, I don't have a strong preference, but I am find with this, e.g.,

new Cesium.Cesium3DTileset

Compared to

new Cesium.Tileset3D

new Cesium.Tiles3D

new Cesium.ThreeDTileset

Cesium.Tileset3D would also be OK. @lilleyse what do you think?

[Patrick Cozzi]

@TomPed I added a section above for you for when you start on 3D Tiles.

[Tom Ped]

Rename Cesium3DTile#readyPromise to Cesium3DTile#contentReadyPromise

@pjcozzi does this mean the field readyPromise in the file Cesium3DTile.js?

[Patrick Cozzi]

@TomPed yes.

[Tom Ped]

Likewise, replace isReplacementRefinable and isContentUnloaded with readonly properties. Complete doc.

@pjcozzi I can't seem to find isReplacementRefinable in the project.

[Sean Lilley]

Ah, don't worry about that we removed that. I'll update the task.

[Tom Ped]

3DTileContentProvider

Should this be Cesium3DTileContentProvider?

[Sean Lilley]

Yeah. The asterisks are just saying all things named like that.

[Tom Ped]

Oh gotcha, so I'll change anything with 3DTileContentProvider to 3DTileContent.

[Sean Lilley]

Yup

[Sean Lilley]

Other than the tests, is Cesium3DTilesetState needed? Can we remove it and just replace it with a _rootRequested boolean in Cesium3DTileset?

It can be removed. I added it in order to support attempting to load tileset.json across multiple frames if the RequestScheduler was full, but that was before I added the RequestScheduler.request function. @pjcozzi I'm more familiar with how this should work, should I handle this one?

[Patrick Cozzi]

@lilleyse sure, whatever you think.

[Tom Ped]

UI elements for loadProgress (commented out) in Cities.html

Are these commented lines to be deleted? Or is the code to be uncommented and fixed?

[Sean Lilley]

We want to bring it back, and add the messages to the UI rather than just the console. Maybe it's best to skip this one for now because @pjcozzi is simplifying 3D Tiles.html on the style branch,

But eventually I think the idea is to have a better UI for visualizing 3D Tiles, similar to the Cesium Inspector demo. This may require looking around at other Sandcastle files for ideas.

[Sean Lilley]

@pjcozzi

Make tests like 'tile visible event is raised' verify that the event is called (check for examples in ModelSpec.js for animation events)

We have a test for tile visible event is raised, is this something different?

[Sean Lilley]

Never mind, so basically just make a spy that checks it was called.

[Tom Ped]

Got it.

[Patrick Cozzi]

Potentially useful reading on V8 GC: https://medium.com/@_lrlna/garbage-collection-in-v8-an-illustrated-guide-d24a952ee3b8

[Rory]

what's the status for vector tiles? it's no update or example, and no spec found in 3D tile 1.0