SLAM系统的关键步骤:构建误差方程,设置优化变量,设置斜方差->Jacobian->Hession矩阵
SLAM系统的关键步骤:构建误差方程,设置优化变量,设置斜方差->Jacobian->Hession矩阵
1、相机的视角相对较低,图像中的场景主要以弱纹理、重复纹理的地面(水泥地、瓷砖),白墙等为主; 2、在室外时,树木的上的特征鲜明,普通提前特征点的方法将会出现大量特征集中在纹理丰富的树木等物体,其余特征相对较少; 3、地面机器人在运动中主要以直行为主(X轴方面的平移),Y轴和Z轴上的运动较少(或因机械震动带来一定的数据值); 4、地面机器人在运动中主要以匀速为主(1m/s至2m/s),起步(停车)时为短暂加速(减速); 5、地面机器人在转弯时经常会发生纯旋转,跟踪过程容易丢失;
1、使用分块方式提取特征点(显示效果与使用非极大值抑制相同,但特征分布不同) 2、使用环形匹配进一步筛选关键帧 a) 先空间后时间匹配,筛选特征点; b) 先时间后空间匹配,筛选特征点。 3、环形匹配过程中使用前向和后向光流 4、优化边缘化处理(舒尔补) 5、增加双目左右视图约束
1、更改globalOptNode.cpp中的void vio_callback(const nav_msgs::Odometry::ConstPtr &pose_msg)中的输入路径,输出位姿数据; 2、将生成轨迹存储成TUM格式的csv,timestamp x y z q_x q_y q_z q_w
VINS-Fusion is an optimization-based multi-sensor state estimator, which achieves accurate self-localization for autonomous applications (drones, cars, and AR/VR). VINS-Fusion is an extension of VINS-Mono, which supports multiple visual-inertial sensor types (mono camera + IMU, stereo cameras + IMU, even stereo cameras only). We also show a toy example of fusing VINS with GPS. Features:
- multiple sensors support (stereo cameras / mono camera+IMU / stereo cameras+IMU)
- online spatial calibration (transformation between camera and IMU)
- online temporal calibration (time offset between camera and IMU)
- visual loop closure
We are the top open-sourced stereo algorithm on KITTI Odometry Benchmark (12.Jan.2019).
Authors: Tong Qin, Peiliang Li, Shaozu Cao, Jie Pan, and Shaojie Shen from the Aerial Robotics Group, HKUST
Authors: Tong Qin, Shaozu Cao, Jie Pan, Peiliang Li, and Shaojie Shen from the Aerial Robotics Group, HKUST
Videos:
Related Papers: (papers are not exactly same with code)
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator, Tong Qin, Peiliang Li, Shaojie Shen, IEEE Transactions on Robotics pdf
@article{qin2017vins,
title={VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator},
author={Qin, Tong and Li, Peiliang and Shen, Shaojie},
journal={IEEE Transactions on Robotics},
year={2018},
volume={34},
number={4},
pages={1004-1020}}
Other papers are under review yet...
If you use VINS-Fusion for your academic research, please cite our related papers.
Related Paper: (paper is not exactly same with code)
-
Online Temporal Calibration for Monocular Visual-Inertial Systems, Tong Qin, Shaojie Shen, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS, 2018), best student paper award pdf
-
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator, Tong Qin, Peiliang Li, Shaojie Shen, IEEE Transactions on Robotics pdf
If you use VINS-Fusion for your academic research, please cite our related papers. bib
Ubuntu 64-bit 16.04 or 18.04. ROS Kinetic or Melodic. ROS Installation
Follow Ceres Installation.
Clone the repository and catkin_make:
cd ~/catkin_ws/src
git clone https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.git
cd ../
catkin_make
source ~/catkin_ws/devel/setup.bash
(if you fail in this step, try to find another computer with clean system or reinstall Ubuntu and ROS)
Download EuRoC MAV Dataset to YOUR_DATASET_FOLDER. Take MH_01 for example, you can run VINS-Fusion with three sensor types (monocular camera + IMU, stereo cameras + IMU and stereo cameras). Open four terminals, run vins odometry, visual loop closure(optional), rviz and play the bag file respectively. Green path is VIO odometry; red path is odometry under visual loop closure.
roslaunch vins vins_rviz.launch
rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml
(optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml
rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag
roslaunch vins vins_rviz.launch
rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion-mobile-robot/config/euroc/euroc_stereo_imu_config.yaml
(optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion-mobile-robot/config/euroc/euroc_stereo_imu_config.yaml
rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag
roslaunch vins vins_rviz.launch
rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion-mobile-robot/config/euroc/euroc_stereo_imu_config.yaml
(optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion-mobile-robot/config/euroc/euroc_stereo_imu_config.yaml
rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag
``` 需要生成轨迹需要在config文件中设置(最好设置完整路径)
生成的轨迹文件有:vio.csv(没有使用回环检测生成的轨迹);loop_vio.csv(发生回环之后轨迹调整之后的轨迹,理论上这个轨迹精度高);vio_loop.csv(使用回环检测之后生成的轨迹)。
<img src="https://github.com/HKUST-Aerial-Robotics/VINS-Fusion/blob/master/support_files/image/euroc.gif" width = 430 height = 240 />
## 4. KITTI Example
### 4.1 KITTI Odometry (Stereo)
Download [KITTI Odometry dataset](http://www.cvlibs.net/datasets/kitti/eval_odometry.php) to YOUR_DATASET_FOLDER. Take sequences 00 for example,
Open two terminals, run vins and rviz respectively.
roslaunch vins vins_rviz.launch
(We evaluated odometry on KITTI benchmark without loop closure funtion)
roslaunch vins vins_rviz.launch
(optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml
rosrun vins kitti_odom_test ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/
Download KITTI raw dataset to YOUR_DATASET_FOLDER. Take 2011_10_03_drive_0027_synced for example. Open three terminals, run vins, global fusion and rviz respectively. Green path is VIO odometry; blue path is odometry under GPS global fusion.
roslaunch vins vins_rviz.launch
rosrun vins kitti_gps_test ~/catkin_ws/src/VINS-Fusion/config/kitti_raw/kitti_10_03_config.yaml YOUR_DATASET_FOLDER/2011_10_03_drive_0027_sync/
rosrun global_fusion global_fusion_node
Download car bag to YOUR_DATASET_FOLDER. Open four terminals, run vins odometry, visual loop closure(optional), rviz and play the bag file respectively. Green path is VIO odometry; red path is odometry under visual loop closure.
roslaunch vins vins_rviz.launch
rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/vi_car/vi_car.yaml
(optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/vi_car/vi_car.yaml
rosbag play YOUR_DATASET_FOLDER/car.bag
VIO is not only a software algorithm, it heavily relies on hardware quality. For beginners, we recommend you to run VIO with professional equipment, which contains global shutter cameras and hardware synchronization.
Write a config file for your device. You can take config files of EuRoC and KITTI as the example.
VINS-Fusion support several camera models (pinhole, mei, equidistant). You can use camera model to calibrate your cameras. We put some example data under /camera_models/calibrationdata to tell you how to calibrate.
cd ~/catkin_ws/src/VINS-Fusion/camera_models/camera_calib_example/
rosrun camera_models Calibrations -w 12 -h 8 -s 80 -i calibrationdata --camera-model pinhole
We use ceres solver for non-linear optimization and DBoW2 for loop detection, a generic camera model and GeographicLib.
To further facilitate the building process, we add docker in our code. Docker environment is like a sandbox, thus makes our code environment-independent. To run with docker, first make sure ros and docker are installed on your machine. Then add your account to docker
group by sudo usermod -aG docker $YOUR_USER_NAME
. Relaunch the terminal or logout and re-login if you get Permission denied
error, type:
cd ~/catkin_ws/src/VINS-Fusion/docker
make build
Note that the docker building process may take a while depends on your network and machine. After VINS-Fusion successfully built, you can run vins estimator with script run.sh
.
Script run.sh
can take several flags and arguments. Flag -k
means KITTI, -l
represents loop fusion, and -g
stands for global fusion. You can get the usage details by ./run.sh -h
. Here are some examples with this script:
# Euroc Monocualr camera + IMU
./run.sh ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml
# Euroc Stereo cameras + IMU with loop fusion
./run.sh -l ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml
# KITTI Odometry (Stereo)
./run.sh -k ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/
# KITTI Odometry (Stereo) with loop fusion
./run.sh -kl ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/
# KITTI GPS Fusion (Stereo + GPS)
./run.sh -kg ~/catkin_ws/src/VINS-Fusion/config/kitti_raw/kitti_10_03_config.yaml YOUR_DATASET_FOLDER/2011_10_03_drive_0027_sync/
In Euroc cases, you need open another terminal and play your bag file. If you need modify the code, simply re-run ./run.sh
with proper auguments after your changes.
We use ceres solver for non-linear optimization and DBoW2 for loop detection, a generic camera model and GeographicLib.
The source code is released under GPLv3 license.
We are still working on improving the code reliability. For any technical issues, please contact Tong Qin <qintonguavATgmail.com>.
For commercial inquiries, please contact Shaojie Shen <eeshaojieATust.hk>.