SJoJoK / 3DGStreamViewer

Tiny 3DGStreamViewer

Tiny 3DGStreamViewer

This is a fork of GaussianSplattingViewer, as a simple viewer for our CVPR 2024 paper 3DGStream.

Note this is not the renderer we used to evaluate the render performance of 3DGStream in our paper "3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos", but is still efficient enough for real-time renderable FVVs

Availble FVVs:

Flame Steak

Usage

Install the dependencies:

pip install -r requirements.txt

Install Pytorch w/ CUDA

Install the diff-gaussian-rasterization following the guidance here.

Install the tiny-cuda-nn.

Install the following package:

pip install cuda-python

Launch the viewer:

python main.py

To view Free-Viewpoint Videos of 3DGStream

1. Unzip the zip file at anywhere you like

   

2. Launch the viewer:

   python main.py
   

3. Click load ply and open the init_3dgs.ply

   

4. Move the camera to a proper position

   

5. Click load FVV and choose the directory where you unzip the FVV

6. Click Step to step into next frame, click Play or Pause to play or pause the FVV, and click Reset to get back to Frame 0

Happy Hacking!

TO-DO List (Not guaranteed)

• View the results of Stage 1 and/or Stage 2
• Support OpenGL-backend
• Align the functionality with the origin repo.

Code-paper discrepancies

1. We discarded the SH rotation, as stated in the paper, due to its costly and unnecessary.
2. The renderer we used to evaluate the render performance is the official SIBR Viewer, which has an highly-optimized OpenGL backend. While, we believe that an open-srouce viewer based on CUDARasterizer is more configurable and helpful for researchers.

Contributing

This project is a tiny viewer designed for simplicity and ease of use. We welcome contributions that aim to improve performance or extend functionality. If you have ideas or improvements, please feel free to submit a pull request.

Acknowledgements

We would like to express our gratitude to the original repository GaussianSplattingViewer for providing the foundation upon which this work is built.