graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"

Home Page:https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question About Discontinuous Color

xerokjames opened this issue · comments

Hello, thanks to your great work! I have a question about the continuity of rendering color.
When I reconstructed a model of a reflective object and rendered it according to my camera path, I found that the colors in some regions were not continuous. In other words, when a camera position made a small change, the colors in these areas had a large jump.
I analyze that this is most likely because of the sorting mechanism. I first found that the transparency of some 3d gaussian points in the reflective regions is close to 0.5, neither high nor low. Sometimes when I change the camera position in a small value, the order of these gaussian points changes, which leads to a jump in the cumulative transmittance, which leads to a jump in color.
The sorting process itself is discontinuous, but is there a continuous way that can achieve the similar effect as sorting? Looking forward to your answers very much!

That's a known limitation of the current algorithm; sorting happens once, based on the primitives' centers, not taking into account the extent, which can lead to these popping artifacts. The "continuous way" (that is, truly multi-view consistent) would be to have a per-pixel sorting step, but that's too costly.
There is this work that tackles this problem exactly, by applying some kind of resorting in a hierarchical manner.