Zhang Chen1,
Zhong Li1,
Liangchen Song2,
Lele Chen1,
Jingyi Yu3,
Junsong Yuan2,
Yi Xu1
1OPPO US Research Center, 2University at Buffalo, 3ShanghaiTech University
Project Page | Paper | Video
This repo is an official PyTorch implementation for the ICCV 2023 paper "NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions". Our work presents a novel type of neural fields with high representation accuracy and model compactness. The repo contains the codes for image fitting, SDF fitting and neural radiance fields.
conda create -n neurbf python=3.9 -y
conda activate neurbf
pip install cupy-cuda117
python -m cupyx.tools.install_library --cuda 11.7 --library cutensor
python -m cupyx.tools.install_library --cuda 11.7 --library cudnn
python -m cupyx.tools.install_library --cuda 11.7 --library nccl
pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117
pip install einops matplotlib kornia imageio imageio-ffmpeg opencv-python pysdf PyMCubes trimesh plotly scipy GPUtil scikit-image scikit-learn pykdtree commentjson tqdm configargparse lpips tensorboard torch-ema ninja tensorboardX numpy pandas rich packaging scipy torchmetrics jax pillow plyfile omegaconf
Build torch-ngp extension
cd thirdparty/torch_ngp/gridencoder
pip install .
cd ../../../
- Ubuntu 20.04 with PyTorch 1.13.0 & CUDA 11.6 on RTX A6000.
- Ubuntu 20.04 with PyTorch 2.0.1 & CUDA 11.7 on RTX A6000.
Download an example pluto image. Put it in data/img
and rename to pluto.png
. Then run
python main.py --config configs/img.py --path ./data/img/pluto.png --alias pluto
The result and tensorboard log will be located in log/img
.
Download the validation set of the DIV2K dataset and put it in data/img/div2k
. The path to each image should be data/img/div2k/DIV2K_valid_HR/xxxx.png
. Then run
python main_img_div2k.py
The script will fit each image separately and the results will be stored in log/img_div2k
.
Download an example armadillo mesh of the Stanford 3D Scanning Repository. Unzip it, put it in data/sdf
, and rename to armadillo.ply
.
Run the following preprocessing script, which normalizes the mesh and sample additional evaluation points.
python preproc_mesh.py --path ./data/sdf/armadillo.ply
Then run
python main.py --config configs/sdf.py --path ./data/sdf/armadillo_nrml.obj --alias armadillo
The result and tensorboard log will be located in log/sdf
.
Download the dataset and unzip to data
. For example, the path to the lego scene should be data/nerf_synthetic/lego
.
For the Synthetic NeRF dataset, use the following command
python main_nerf.py --config_init configs/nerf_tensorf/nerf_synthetic_init.py --config configs/nerf_tensorf/nerf_synthetic.py --data_name lego
It will first distill scene information to initialize RBF position and shape parameters, and then train the full model. The result and tensorboard log will be located in log/nerf_synthetic
. Change --data_name
to run on other scenes.
If you find our work useful, please consider citing:
@inproceedings{chen2023neurbf,
title={NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions},
author={Chen, Zhang and Li, Zhong and Song, Liangchen and Chen, Lele and Yu, Jingyi and Yuan, Junsong and Xu, Yi},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
year={2023}
}