GIRA3D Reconstruction

This component of GIRA3D enables compact point cloud modeling for depth-intensity reconstruction. The point cloud modeling is performed via Self-Organizing Gaussian Mixture Modeling (SOGMM) presented in:

[1] K. Goel, N. Michael, and W. Tabib, “Probabilistic Point Cloud Modeling via Self-Organizing Gaussian Mixture Models,” IEEE Robotics and Automation Letters, vol. 8, no. 5, pp. 2526–2533, May 2023, doi: 10.1109/LRA.2023.3256923.

@ARTICLE{goel_probabilistic_2023,
  title = {Probabilistic Point Cloud Modeling via Self-Organizing Gaussian Mixture Models},
  author = {Goel, Kshitij and Michael, Nathan and Tabib, Wennie},
  year = {2023},
  month = may,
  journal={IEEE Robotics and Automation Letters},
  volume = {8},
  number = {5},
  pages = {2526--2533},
  issn = {2377-3766},
  doi = {10.1109/LRA.2023.3256923},
  keywords = {Adaptation models,Complexity theory,Computational modeling,Data models,field robots,Mapping,Point cloud compression,Probabilistic logic,RGB-D perception,Three-dimensional displays}
}

A video demonstrating the features of SOGMM can be found here.

If you use this codebase in your work, please consider citing the article above.

Overview

This repository is a meta workspace containing two colcon workspaces:

The dry workspace contains dependencies that we choose to compile from their source code.
The wet workspace contains the main codebase.

The dry workspace contains the following git submodules:

eigen_colcon: Colcon package to pull Eigen 3.4.
pybind11_colcon: Colcon package to pull pybind11.
open3d_colcon: Colcon package to pull our modifications to Open3D.

Note open3d_colcon will not be compiled for macOS.

The wet workspace contains the following git submodules:

self_organizing_gmm: C++ codebase accelerated with OpenMP for Intel/AMD/ARM CPUs.
sogmm_open3d: C++ codebase accelerated with CUDA and Open3D for Intel/AMD/ARM CPUs and NVIDIA GPUs.

Note sogmm_open3d will not be compiled for macOS.

sogmm_py: Scripts for visualization, comparison, testing, and profiling.

Supported Platforms

The codebase has been tested on the following platforms:

Linux

Platform ID	OS	CPU	GPU	Memory (CPU/GPU)	CUDA Version
Ryzen/RTX3090	Ubuntu 20.04	AMD Ryzen Threadripper 3960X, 48 threads	NVIDIA GeForce RTX 3090	252 GB/24 GB	<= 12.1
Intel/RTX3060	Ubuntu 20.04	Intel Core i9-10900K, 20 threads	NVIDIA GeForce RTX 3060	32 GB/12 GB	<= 12.1
ARM-12c/Orin	Ubuntu 20.04	ARMv8 Processor rev 1 (v8l), 12 threads	NVIDIA Jetson Orin	32 GB	<= 11.4
ARM-8c/Xavier	Ubuntu 20.04	ARMv8 Processor rev 0 (v8l), 8 threads	NVIDIA Jetson Xavier	16 GB	<= 11.4
ARM-6c/TX2	Ubuntu 20.04	ARM Cortex-A57, 6 threads	NVIDIA Jetson TX2	8 GB	<= 10.1

macOS

Note CUDA-dependent repository, sogmm_open3d, is not supported for macOS.

Platform ID	OS	CPU	GPU	Memory (CPU/GPU)	CUDA Version
MacBook/Intel	MacOS Ventura 13.3.1	2.3 GHz 8-Core Intel Core i9	N/A	16 GB	N/A

Build and Install

We require using a Python 3.8 virtual environment using pip. Assume that the virtual environment is created within this repository using the folder name .venv. First, we set up the colcon build environment.

git clone git@github.com:gira3d/gira3d-reconstruction.git
git checkout master # or your favorite branch
git submodule update --recursive --init
cd gira3d-reconstruction
python3.8 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install colcon-common-extensions wheel

For Python scripts, run-time dependencies can be installed using pip. Make sure the virtual environment created above is active.

pip install numpy scikit-learn opencv-python scikit-image future matplotlib termcolor

Finally, run the build script:

./build

Some helper scripts are provided as follows:

clean: Clean the builds for both dry and wet workspaces.
configure: Pull the submodules.

Warning If you run into issues with OpenMP on MacOS, you might need to use the llvm clang compiler as opposed to the Apple Clang compiler.

Environment Setup

These steps also compile the python bindings. To use these python interfaces, first run:

workon

Note For NVIDIA Jetson platforms (i.e., Orin, Xavier, and TX2), run workon_jetson.

User Guide

pip install mkdocs mkdocs-material
mkdocs serve

You should be able to view the documentation at this level over at localhost.

Utilize this documentation for tutorials and examples that use one or many of the submodules.

Note: The documentation is stored using git lfs. You may need to run the following in order to make sure it is pulled correctly:

cd docs
git lfs install
git lfs pull

Authors

Kshitij Goel, Wennie Tabib

Acknowledgments

ndt_map: BSD License, Copyright (c) 2010, AASS Research Center, Orebro University.
octomap: New BSD License, Copyright (c) 2009-2013, K.M. Wurm and A. Hornung, University of Freiburg.
Eigen: Mozilla Public License, v. 2.0.

License

BSD 3-Clause License

Copyright (c) 2023, Kshitij Goel, Wennie Tabib

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
   list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its
   contributors may be used to endorse or promote products derived from
   this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Epsilon8854 / gira3d-reconstruction