eriksandstroem / Loopy-SLAM

Loopy-SLAM: Dense Neural SLAM with Loop Closures

[CVPR 2024]

Lorenzo Liso* . Erik Sandström* · Vladimir Yugay · Luc Van Gool · Martin R. Oswald

(* Equal Contribution)

Paper | Project Page

Loopy-SLAM produces accurate dense geometry and camera tracking on large-scale indoor scenes.

Loopy-SLAM Architecture.

Table of Contents

1. Installation
2. Run
3. Evaluation
4. Acknowledgement
5. Citation
6. Contact

Installation

First you have to make sure that you have all dependencies in place. The simplest way to do so is to use anaconda.

If you are running Loopy-SLAM on a cluster GPU without a display, we recommend installing the headless version of Open3D which is required to evaluate the depth L1 metric of the reconstructed mesh. This requires compiling Open3D from scratch. The code is tested on Open3D 15.1 and 16.0. If compiling Open3D from scratch, remove the Open3D dependency from the env.yaml file.

You can create an anaconda environment called loopy-slam.

conda env create -f env.yaml
conda activate loopy-slam

For evaluating the F-score, download and install this library using pip.

git clone https://github.com/tfy14esa/evaluate_3d_reconstruction_lib.git
cd evaluate_3d_reconstruction_lib
pip install .

To install the bag of words place recognition library, please follow the instructions here.

Data Download

Replica

Download the data as below and the data is saved into the ./datasets/Replica folder. Note that the Replica data is generated by the authors of iMAP (but hosted by the authors of NICE-SLAM). Please cite iMAP if you use the data.

bash scripts/download_replica.sh

To be able to evaluate the reconstruction error, download the ground truth Replica meshes where unseen region have been culled.

bash scripts/download_cull_replica_mesh.sh

TUM-RGBD

bash scripts/download_tum.sh

DATAROOT is ./datasets by default. Please change the input_folder path in the scene specific config files if stored somewhere else on your machine.

ScanNet

Please follow the data downloading procedure on the ScanNet website, and extract color/depth frames from the .sens file using this code.

[Directory structure of ScanNet (click to expand)]

DATAROOT is ./datasets by default. If a sequence (sceneXXXX_XX) is stored in other places, please change the input_folder path in the config file or in the command line.

  DATAROOT
  └── scannet
        └── scene0000_00
            └── frames
                ├── color
                │   ├── 0.jpg
                │   ├── 1.jpg
                │   ├── ...
                │   └── ...
                ├── depth
                │   ├── 0.png
                │   ├── 1.png
                │   ├── ...
                │   └── ...
                ├── intrinsic
                └── pose
                    ├── 0.txt
                    ├── 1.txt
                    ├── ...
                    └── ...

We use the following sequences:

scene0000_00
scene0025_02
scene0059_00
scene0062_00
scene0103_00
scene0106_00
scene0126_00
scene0169_00
scene0181_00
scene0207_00

Run

For running Loopy-SLAM, we recommend using weights and biases for the logging. This can be turned on by setting the wandb flag to True in the configs/point_slam.yaml file. Also make sure to specify the path wandb_folder. If you don't have a wandb account, first create one. Each scene has a config folder, where the input_folder and output paths need to be specified. Below, we show some example run commands for one scene from each dataset. If you use a batch processing system (e.g SLURM), you might find our repro.sh script useful.

Replica

To run Loopy-SLAM on the room0 scene, run the following command.

python run.py configs/Replica/room0.yaml

After reconstruction, the trajectory error will be evaluated and so will the mesh accuracy along with the rendering metrics.

TUM-RGBD

To run Loopy-SLAM on the freiburg1_desk scene, run the following command.

python run.py configs/TUM_RGBD/freiburg1_desk.yaml

After reconstruction, the trajectory error will be evaluated automatically.

ScanNet

To run Loopy-SLAM on the scene0000_00 scene, run the following command.

python run.py configs/ScanNet/scene0000_00.yaml

After reconstruction, the trajectory error will be evaluated automatically.

Acknowledgement

Our codebase is based on Point-SLAM.

Reproducibility

There may be minor differences between the released codebase and the results reported in the paper. Further, we note that the GPU hardware has an influence, despite running the same seed and conda environment.

Citation

If you find our code or paper useful, please cite

@inproceedings{liso2024loopy,
  title={Loopy-slam: Dense neural slam with loop closures},
  author={Liso, Lorenzo and Sandstr{\"o}m, Erik and Yugay, Vladimir and Van Gool, Luc and Oswald, Martin R},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={20363--20373},
  year={2024}
}

Contact

Contact Erik Sandström for questions, comments and reporting bugs.