Real-time Semantic Reconstruction
This project is built for the perception system of an autonomous disinfection robot. The core components of this module is in charge for SLAM and 3D semmantic reconstruction.
Dependancy
- Build System
- C++17
- CMake >= 3.18
- CUDA >= 9.0
- Externel Libraries
- OpenVSLAM (follow the official installation guide but with this custom branch)
- ZED SDK (optional)
Note
For installing externel dependancies, to avoid collision with existing system packages, it is
recommended to install all packages (including the ones that OpenVSLAM depends on) into
a local directory such as ~/.local
instead of the system path /usr/local
. This can
be easily achieved by adding -DCMAKE_INSTALL_PREFIX=$HOME/.local
option to a CMake command.
Another benifit of this is that sudo
privilege is not needed for the installation.
Compilation
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
Running SLAM
ZED 1
To run without ZED SDK (stereo calibration is required,
see example in configs/zed_native_stereo.yaml
)
./run_zed_native -v <path_to_vocab.dbow2> -c ../configs/zed_native_stereo.yaml --devid 0
To run with ZED SDK (CUDA and ZED SDK is required)
./run_zed -v <path_to_vocab.dbow2> -c ../configs/zed_stereo.yaml
Intel SR300
./run_sr300 -v <path_to_vocab.dbow2> -c ../configs/sr300_rgbd.yaml --depth
Running Reconstruction
This section is still WIP
TODO
- Add logic to not update TSDF when tracking is lost
- Update installation.md
- Add a sciprt folder for easier user access
- Update segmentation inference example to take in custom images
- Add rotation arthimetics (e.g. rotation matrices interpolation)
- Use pre-processer to allow online.cc to compile without libtorch
- TensorRT for saving GPU memory