Neural Density-Distance Fields

Project Page | arXiv

PyTorch implementation of Neural Distance-Density Field (NeDDF), a 3D representation that reciprocally constrains the distance and density fields.

Neural Density-Distance Fields
Itsuki Ueda¹, Yoshihiro Fukuhara², Hirokatsu Kataoka³, Hiroaki Aizawa⁴, Hidehiko Shishido¹, Itaru Kitahara¹
¹University of Tsukuba, ²Waseda University, ³National Institute of Advanced Industrial Science and Technology (AIST), ⁴Hiroshima University
in ECCV 2022(poster)

Prerequisite

docker
docker-compose
nvidia-docker2

How to setup docker container

This repository is based on Ascender project. Please refer to Ascender for detailed instructions on how to set up the host environment.

## Move to the directory where docker-compose.yaml exists.
$ cd /path/to/neddf/environments/gpu

## build and start docker container
$ docker compose up -d core

## Enter docker container with bash
$ docker compose exec core bash

How to run code

You can download the nerf_synthetic_dataset and nerf_llff_dataset proposed in original NeRF paper here. Please place downloaded data to data/

Train NeDDF

In the default configuration, the bunny_smoke dataset is set to learn.

Run

$ poetry run python neddf/scripts/run.py

To use other dataset with data format of nerf_synthetic, please select other dataset directory like follow:

$ poetry run python neddf/scripts/run.py dataset.dataset_dir="data/nerf_synthetic/drums/"

Logs are saved in outputs/{datatime}/. Model parameters are saved to outputs/{datatime}/models/. Rendering results are saved to outputs/{datatime}/render/, images rendered from a camera is in iteration number directory, and visualization of field slices are in fields directory.

Evaluate NeDDF

Run

$ poetry run python neddf/scripts/run_eval.py {pretrained files directory}

For example, use following commands to evaluate neddf model in bunny_smoke scene.

$ poetry run python neddf/scripts/run_eval.py pretrained/bunny_smoke/

You can download pretrained models of nerf_synthetic_dataset from this link

Visualize pretrained fields

To visualize trained distance, density, aux.gradient and color fields, please run following command.

$ poetry run python neddf/scripts/fields_visualizer.py {pretrained files directory}

This visualizer draw fields by 2D slices. Following video is visualization example in lego scene(pretrained/lego)

visualizer_fields.mp4

Visualize for check dataset

To visualize dataset, please run following command (override of dataset config is optional):

$ poetry run python neddf/scripts/dataset_visualizer.py dataset.dataset_dir=data/bunny_smoke/

ueda0319 / neddf