Pytorch implementation of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (Mildenhall et al., ECCV 2020 Oral, Best Paper Honorable Mention) and its extension to follow-up research projects.
 |
|---|
| NeRF Overview. Figure from the project page of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, Mildenhall et al., ECCV 2020. |
To configure the Python virtual environment and install dependencies, open the shell and run the following commands:
# Clone the repository
git clone https://github.com/DveloperY0115/torch-NeRF.git
cd torch-NeRF
# Create virtual environment
virtualenv venv -p=3.8
source venv/bin/activate
# Install dependencies
pip3 install -r requirements.txt
You may use any tools of your preference such as conda. Our codes should reproduce the results regardless of which dependency management tool is used as long as the same versions of dependencies are installed.
To download the synthetic dataset for training, issue the following command in the shell:
sh scripts/data/download_example_data.sh
This should create data directory under the project root (torch-NeRF by default) and download datasets provided by the authors of NeRF (ECCV 2020).
The configuration is set for lego scene included in Blender dataset. Executing the following command will initiate training:
python torch_nerf/runners/run_train.py
Once a scene representation is trained, it can be rendered using the script run_render.py under torch_nerf/runners directory. Note that you need to specify the path to the checkpoint file by modifying the file under config/train_params.
The rendering script stores rendered images in render_out directory under the project root. To create a video from the collection of consecutive frames, use the script scripts/utils/create_video.py.
NOTE: All images shown below are reproducible using our codes.
- Implement the base model and volume rendering pipeline proposed in NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (ECCV 2020).
- Extend the implementation to Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields (ICCV 2021).
- Extend the implementation to Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields (CVPR 2022).
- Extend the implementation to NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (CVPR 2022).
- Extend the implementation to Neural 3D Scene Reconstruction with the Manhattan-world Assumption (CVPR 2022).
- Extend the implementation to Urban Radiance Fields (CVPR 2022).
- Extend the implementation to Block-NeRF: Scalable Large Scene Neural View Synthesis (CVPR 2022).
- Adapt techniques for accelerating training & inference. The selected candidates are listed below (subject to change):