Support for loading large GLB files without copying entire file to managed memory

[Stephen Lucas]

An issue with the current GltfImport.LoadGltfBinary implementation is that it requires the entire file to have been loaded into a managed byte array for the duration of the load. This is problematic with larger files on devices that have limited memory and can't afford a 2X temporary memory overhead during loading, particularly when things like KTX texture extensions are being used and the file also contains redundant PNG/JPG texture data for compatibility.

I assume the use of Stream to fetch data incrementally from an abstract source was avoided in the implementation for performance reasons, but one possible solution to this problem would be to use memory mapped files on platforms that support them. This could be supported by switching the input type from byte[] to ReadOnlyMemory and ReadOnlySpan (depending on async or sync method usage) in Unity 2021+ which abstracts the source of the memory. Although this does limit the maximum filesize to 2GB vs. the 4GB limit of GLB without a more involved view chunking implementation.

I'm curious if any other approaches to this issue have been considered previously?

Thanks!