yuzhou42 / disinfect-slam

Real-time Semantic Reconstruction

This project is built for the perception system of an autonomous disinfection robot. The core components of this module is in charge for SLAM and 3D semmantic reconstruction.

Dependancy

• Build System
  • C++17
  • CMake >= 3.18
    1. pip install --upgrade cmake
    2. source ~/.bashrc
  • CUDA >= 9.0
    1. sudo apt install nvidia-cuda-toolkit
• External Libraries
  • OpenVSLAM (follow the official installation guide but with this custom branch)
  • ZED SDK (optional)
  • glfw3 (alternatively, sudo apt-get install libglfw3-dev)
  • realsense2

Note

For installing externel dependancies, to avoid collision with existing system packages, it is recommended to install all packages (including the ones that OpenVSLAM depends on) into a local directory such as ~/.local instead of the system path /usr/local. This can be easily achieved by adding -DCMAKE_INSTALL_PREFIX=$HOME/.local option to a CMake command. Another benifit of this is that sudo privilege is not needed for the installation.

Compilation

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..

Running SLAM

ZED 1

To run without ZED SDK (stereo calibration is required, see example in configs/zed_native_stereo.yaml)

./run_zed_native -v <path_to_vocab.dbow2> -c ../configs/zed_native_stereo.yaml --devid 0

To run with ZED SDK (CUDA and ZED SDK is required)

./run_zed -v <path_to_vocab.dbow2> -c ../configs/zed_stereo.yaml

Intel SR300

./run_sr300 -v <path_to_vocab.dbow2> -c ../configs/sr300_rgbd.yaml --depth

Running Reconstruction

This section is still WIP

Running ROS

installtion

1. rviz_visual_tools
2. TSDF2Mesh

setup

• add libtorch

ln -s /path_to_torchlib third_party/libtorch

• setup dependencies

sudo apt-get install ros-noetic-eigen-stl-containers
sudo apt-get install ros-noetic-graph-msgs

• set model_path/calib_path/orb_vocab_path in launch/disinfslam.launch

• we are only building the tsdf example, so comment the other examples out of CMakeLists.txt

• compile

catkin build  

Running

roslaunch disinfslam disinfslam.launch 
roslaunch TSDF2Mesh tsdf2mesh.launch

Visulization

rviz
# open recent config and selet ~/slam_ws/slam.rviz

TODO

• Add logic to not update TSDF when tracking is lost
• Update installation.md
• Add a sciprt folder for easier user access
• Update segmentation inference example to take in custom images
• Add rotation arthimetics (e.g. rotation matrices interpolation)
• Use pre-processer to allow online.cc to compile without libtorch
• TensorRT for saving GPU memory

1. Install Nvidia drivers, CUDA (11.x), nvcc
2. Install ROS
3. Install Klampt
4. Install ORB_SLAM3
5. pip install opencv-contrib-python
6. sudo apt-get install ros-noetic-rviz-visual-tools
7. Copy to ws TSDF2Mesh
8. Copy to ws disinfect_slam
9. Install librealsense2
10. Install ZED3 (or remove dependencies)