wbhu / Tri-MipRF

Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields, ICCV'23 (Oral, Best Paper Finalist)

Home Page:https://wbhu.github.io/projects/Tri-MipRF

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About the use of custom data resulting in insufficient CPU memory and poor performance

Flyingdog-Huang opened this issue · comments

Data resolution: 1924*1080
Quantity: 650 pieces
Type: jpg (three-channel)

Problem 1: When using the data conversion script convert_blender_data.py, the CPU-mem must be 42GB
Temporary solution: Reduce the number, reduce the resolution
Q: how can I reduce CPU mem?

Problem 2: Poor effect (PSNR<16, loss>0.004)
Detailed description:
In the data of 1920 * 1072 - 163
image

In the data of 800 * 800 - 325
image
Q: how can I improve model and rending effect ?

Q: how can I reduce CPU mem?
A: The current implementation caches all rays and pixels in CPU memory for efficiency consideration. You may modify it if your machine doesn't have enough memory.

Q: how can I improve model and rending effect ?
A: from your photos, the scene seems unbounded, if so, this model may not be suitable as it is designed for bounded data, if not, you may set a suitable AABB to bound your scene

You may need to modify 'render_bkgd' for the custom dataset. Pose may not be correct as well - if you use colmap, then it follows opencv - not opengl.