Giters

aipixel / GPS-Gaussian

[CVPR 2024 Highlight] The official repo for “GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis”

Home Page:https://shunyuanzheng.github.io/GPS-Gaussian

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Training on Multi-View RGB-D Data and Testing on Single-View RGB-D

GuanxingLu opened this issue · comments

commented

Hi there! I'm impressed with your work and had a question about the possibility of training a model on multi-view RGB-D data and then testing it on single-view RGB-D data. This scenario is particularly useful when training in a simulation environment and deploying the model in the real world.

I would greatly appreciate any insights or guidance on how to proceed with this. Thank you for your time and assistance!

commented

Hi,

I think when using RGB-D data as input, it becomes a task of extrapolation rather than an interpolation problem under a stereo camera setting. If there is slight self-occlusion, I think it can synthesize high-quality novel views under a limited pitch & roll. However, if severe self-occlusion exists, I think you need an auxiliary network to complement the incomplete Gaussian points or an inpainting network to do it on 2D novel view outputs.