Figure 1. Different corner cases for SLAM
1. All the rosbag files have been released now, while the GT, calibration files and robot size parameters will be made public upon paper acceptance. All the experiment results in the paper can be reproduced.
2. To get the calibration and GT files in advance, you can contact us at 1195391308@qq.com. If our work is helpful for your research, please follow us and give a star
We introduce Ground-Challenge: a novel dataset collected by a ground robot with multiple sensors including an RGB-D camera, an inertial measurement unit (IMU), a wheel odometer and a 3D LiDAR to support the research on corner cases of visual SLAM systems. Our dataset comprises 36 trajectories with diverse corner cases such as aggressive motion, severe occlusion, changing illumination, few textures, pure rotation, motion blur, wheel suspension, etc. Some state-of-the-art SLAM algorithms are tested on our dataset, showing that these systems are seriously drifting and even failing on specific sequences. We will release the dataset and relevant materials upon paper publication to benefit the research community.
- We collect a novel visual SLAM dataset for ground robots with a rich pool of sensors in diverse environments both indoors and outdoors. Particularly, the dataset covers a series of challenging sequences for sensor failures and specific movement patterns.
- State-of-the-art SLAM algorithms of different settings are tested on our benchmark. And the results indicate these systems are not robust enough for situations such as sensor failures.
- To facilitate the research on corner cases of robot navigation, we will release the dataset with ground truth trajectories and the configuration file of each tested algorithm upon paper publication.
The ground robot is given below. The unit of the figures is centimeter.
Figure 2. The data capture robot.
All the sensors and track devices and their most important parameters are listed as below:
-
LIDAR Velodyne VLP-16, 360 Horizontal Field of View (FOV),-30 to +10 vertical FOV,10Hz,Max Range 200 m,Range Resolution 3 cm, Horizontal Angular Resolution 0.2°.
-
V-I Sensor,Realsense d435i,RGB/Depth 640*480,69H-FOV,42.5V-FOV,15Hz;IMU 6-axix, 200Hz
-
IMU,Xsens Mti-300,9-axis,400Hz;
-
Wheel Odometer,AgileX,2D,25Hz;
The rostopics of our rosbag sequences are listed as follows:
-
LIDAR:
/velodyne_points
-
V-I Sensor:
/camera/color/image_raw
,
/camera/depth/image_raw
,
/camera/imu
-
IMU:
/imu/data
-
Wheel Odometer:
/odom
An overview of Ground-Challenge is given in the table below:
Scenario | Darkroom | Occlusion | Office | Room | Wall | Motionblur | Hall | Loop | Roughroad | Corridor | Rotation | Static | Slope | TOTAL |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Number | 3 | 4 | 3 | 3 | 3 | 3 | 3 | 2 | 3 | 2 | 3 | 2 | 2 | 36 |
Dist/m | 92.0 | 273.8 | 75.5 | 102.1 | 86.7 | 166.6 | 236.3 | 371.8 | 68.1 | 164.3 | 12.4 | 1.9 | 128.5 | 1780.0 |
Duration/s | 203.6 | 334.2 | 164.0 | 154.7 | 189.3 | 145.5 | 302.4 | 332.7 | 186.3 | 198.1 | 183.2 | 92.6 | 195.0 | 2681.6 |
Size/GB | 6.1 | 9.9 | 4.7 | 4.6 | 5.6 | 4.3 | 8.7 | 9.9 | 5.4 | 5.8 | 5.4 | 2.7 | 5.7 | 78.8 |
Sequence Name | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|
Darkroom1 | 2.9g | 100s | slight light, going into a room | Rosbag | GT |
Darkroom2 | 2.3g | 76s | sharp turn | Rosbag | GT |
Darkroom3 | 1.9g | 64s | slight light | Rosbag | GT |
Occlusion1 | 2.9g | 97s | moving feet, far features | Rosbag | GT |
Occlusion2 | 3.2g | 108s | hand occlusion | Rosbag | GT |
Occlusion3 | 2.6g | 89s | hand occlusion | Rosbag | GT |
Occlusion4 | 1.2g | 40s | complete occlusion | Rosbag | GT |
Office1 | 1.3g | 46s | exposure change | Rosbag | GT |
Office2 | 1.9g | 66s | going into a dark room | Rosbag | GT |
Office3 | 1.5g | 52s | office | Rosbag | GT |
Room1 | 1.3g | 46s | exposure change | Rosbag | GT |
Room2 | 1.9g | 66s | going into a dark room | Rosbag | GT |
Room3 | 1.5g | 52s | office | Rosbag | GT |
Motionblur1 | 1.5g | 52s | aggressive motion | Rosbag | GT |
Motionblur2 | 1.6g | 54s | aggressive motion | Rosbag | GT |
Motionblur3 | 1.2g | 40s | aggressive motion | Rosbag | GT |
Wall1 | 1.7g | 59s | wall in a corridor | Rosbag | GT |
Wall2 | 2.0g | 66s | wall in a big hall | Rosbag | GT |
Wall3 | 3.9g | 65s | wall in a corridor | Rosbag | GT |
Sequence Name | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|
Hall1 | 2.6g | 91s | slippery ground, a reflective surface | Rosbag | GT |
Hall2 | 3.2g | 110s | slippery ground, a reflective surface | Rosbag | GT |
Hall3 | 2.9g | 101s | slippery ground, walking human | Rosbag | GT |
Loop1 | 4.1g | 97s | moving feet, far features | Rosbag | GT |
Loop2 | 5.8g | 137s | hand occlusion | Rosbag | GT |
Roughroad1 | 2.2g | 75s | rough road | Rosbag | GT |
Roughroad2 | 1.5g | 52s | rough road | Rosbag | GT |
Roughroad3 | 1.8g | 59s | rough road | Rosbag | GT |
Sequence Name | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|
Corridor1 | 2.9g | 100s | zigzag, long corridor | Rosbag | GT |
Corridor2 | 2.9g | 98s | straight forward, long corridor | Rosbag | GT |
Rotation1 | 1.6g | 53s | moving feet, far features | Rosbag | GT |
Rotation2 | 2.1g | 73s | hand occlusion | Rosbag | GT |
Rotation3 | 1.7g | 57s | rough road | Rosbag | GT |
Static1 | 1.6g | 56s | rough road | Rosbag | GT |
Static2 | 1.1g | 37s | rough road | Rosbag | GT |
Slope1 | 2.8g | 96s | slope | Rosbag | GT |
Slope2 | 2.9g | 99s | slope | Rosbag | GT |
For convenience of evaluation, we provide configuration files of some well-known SLAM systems as below: