Neural Implicit Surface Reconstruction using Imaging Sonar

This repo contains the source code for the ICRA 2023 paper Neural Implicit Surface Reconstruction using Imaging Sonar

Ground Truth Mesh	Our Reconstruction	Volumetric Albedo	Back-projection

Usage

Anaconda Environment

Create a virtual python environment using Anaconda:

conda create -n neusis python=3.7
conda activate neusis

Setup

Install PyTorch then:

 git clone https://github.com/rpl-cmu/neusis.git
 cd neusis
 pip install -r requirements.txt

Data

Sample datasets are available for download here datasets. Unzip the files inside the data directory.
The data is organized as follows:

data/<dataset_name>
|-- Data
    |-- <pose 1>.pkl        # data for each view (includes the sonar image and pose)
    |-- <pose 2>.pkl 
    ...
|-- Config.json      # Sonar configuration

Running

Training:

python run_sdf.py --conf confs/<dataset_name>.conf

Example: python run_sdf.py --conf confs/14deg_submarine.conf

The resulting meshes are saved in the following directory experiments/<dataset_name>/meshes. The parameter val_mesh_freq in confs/<dataset_name>.conf controls the frequency of mesh generation.

Notes on training

We used an NVIDIA 3090 GPU for training. Depending on available compute, consider adjusting the following parameters in confs/<dataset_name>.conf:

Parameter	Description
arc_n_samples	number of samples along each arc
num_select_pixels	number of sampled pixels (px)
percent_select_true	percentage of px with intensity > `ε` that are be selected every training iteration
n_samples	number of samples along each acoustic ray

Depending on weight initialization, we noticed that the network might converge to "bad" local minimas. A quick way to make sure that the network is training correctly is to check that the intensity Loss is decreasing after a handful of epochs. Otherwise, please restart the training.

This issue could be mitigated via data normalization and enabling geometric initialization of network weights: Issue

Citation

Consider citing as below if you find our work helpful to your project:

@inproceedings{qadri2023neural,
  title={Neural implicit surface reconstruction using imaging sonar},
  author={Qadri, Mohamad and Kaess, Michael and Gkioulekas, Ioannis},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={1040--1047},
  year={2023},
  organization={IEEE}
}

Acknowledgement

Some code snippets are borrowed from IDR, and NeuS. Thanks for these projects!

huangjuite / neusis