Figure 1. Sample Images
Intelligent ground robots have significant application prospects due to their capability to fulfill special missions in complex and dangerous environments.Simultaneous Localization and Mapping (SLAM) is a critical module of robot navigation.Although the research of SLAM has achieved great progress in the past decades,it still has a long way towards mature deployment.Unfortunately, most of the existing datasets of ground robots have limited scenes and scarce categories of sensors, which restricts the development of localization algorithms for ground robots in variable and complex real-world environments,especially in darkness,fog,sharp turns. Here we introduce M2DGR:a new large-scale dataset collected by a ground robot which contains a full sensor-suite include surrounding and sky-pointing RGB cameras,an infrared camera,an event camera,an inertial measurement unit(IMU),a LIDAR,a consumer grade Global Navigation Satellite System (GNSS) receiver and a GNSS-IMU navigation system with real-time kinematic (RTK) GNSS suite providing centimetre accurate global positioning.The dataset include ample trajectories in highly diverse scenes covering rooms, elevators, streets, parking lots and so on.
Based on M2DGR, we evaluate the performance of a few state-of-the-art of SLAM algorithms in a few real-world environments, and analyze the scenarios where ground robots are not robust to.For the benefit of the research community,we make the dataset and evaluation results public.Our dataset,development toolkits and more infomation are now available through the link in the footnote.
Keywords:Dataset, Multi-model, Multi-scenario,Ground Robot
- We collected long-term challenging sequences for ground robots both indoors and outdoors with a complete sensor suite, which includes six surround-view fish-eye cameras, a sky-pointing fish-eye camera, a perspective color camera, an event camera, an infrared camera, a 32-beam LIDAR, two GNSS receivers, and two IMUs. To our knowledge, this is the first SLAM dataset focusing on ground robot navigation with such rich sensory information.
- We recorded trajectories in a few challenging scenarios like lifts, complete darkness, which can easily fail existing localization solutions. These situations are commonly faced in ground robot applications, while they are seldom discussed in previous datasets.
- We launched a comprehensive benchmark for ground robot navigation. On this benchmark, we evaluated existing state-of-the-art SLAM algorithms of various designs and analyzed their characteristics and defects individually.
Physical drawings and schematics of the ground robot is given below. The unit of the figures is centimeter.
Figure 2. The GAEA Ground Robot Equipped with a Full Sensor Suite.The directions of the sensors are marked in different colors,red for X,green for Y and blue for Z.
Figure 3. Side View of GAEA ROBOT. Unit: cm
Figure 4. Middle Layer of GAEA ROBOT. Unit: cm
Figure 5. Top Layer of GAEA ROBOT. Unit: cm
All the sensors and track devices and their most important parameters are listed as below:
-
LIDAR Velodyne VLP-32C, 360 Horizontal Field of View (FOV),-30 to +10 vertical FOV,10Hz,Max Range 200 m,Range Resolution 3 cm, Horizontal Angular Resolution 0.2°.
-
RGB Camera FLIR Pointgrey CM3-U3-13Y3C-CS,fish-eye lens,1280*1024,190 HFOV,190 V-FOV, 15 Hz
-
GNSS Ublox M8T, GPS/BeiDou, 1Hz
-
Infrared Camera,PLUG 617,640*512,90.2 H-FOV,70.6 V-FOV,25Hz;
-
V-I Sensor,Realsense d435i,RGB/Depth 640*480,69H-FOV,42.5V-FOV,15Hz;IMU 6-axix, 200Hz
-
Event Camera Inivation DVXplorer, 640*480,15Hz;
-
IMU,Handsfree A9,9-axis,150Hz;
-
GNSS-IMU Xsens Mti 680G. GNSS-RTK,localization precision 2cm,100Hz;IMU 9-axis,100 Hz;
-
Laser Scanner Leica MS60, localization 1mm+1.5ppm
-
Motion-capture System Vicon Vero 2.2, localization accuracy 1mm, 50 Hz;
The rostopics of our rosbag sequences are listed as follows:
-
LIDAR:
/velodyne_points
-
RGB Camera:
/camera/left/image_raw/compressed
,
/camera/right/image_raw/compressed
,
/camera/third/image_raw/compressed
,
/camera/fourth/image_raw/compressed
,
/camera/fifth/image_raw/compressed
,
/camera/sixth/image_raw/compressed
,
/camera/head/image_raw/compressed
-
GNSS Ublox M8T:
/ublox/aidalm
,
/ublox/aideph
,
/ublox/fix
,
/ublox/fix_velocity
,
/ublox/monhw
,
/ublox/navclock
,
/ublox/navpvt
,
/ublox/navsat
,
/ublox/navstatus
,
/ublox/rxmraw
-
Infrared Camera:
/thermal_image_raw
-
V-I Sensor:
/camera/color/image_raw/compressed
,
/camera/imu
-
Event Camera:
/dvs/events
,
/dvs_rendering/compressed
-
IMU:
/handsfree/imu
We make public ALL THE SEQUENCES with their GT now.
Figure 6. A sample video with fish-eye image(both forward-looking and sky-pointing),perspective image,thermal-infrared image,event image and lidar odometry
An overview of M2DGR is given in the table below:
Scenario | Street | Circle | Gate | Walk | Hall | Door | Lift | Room | Roomdark | TOTAL |
---|---|---|---|---|---|---|---|---|---|---|
Number | 10 | 2 | 3 | 1 | 5 | 2 | 4 | 3 | 6 | 36 |
Size/GB | 590.7 | 50.6 | 65.9 | 21.5 | 117.4 | 46.0 | 112.1 | 45.3 | 171.1 | 1220.6 |
Duration/s | 7958 | 478 | 782 | 291 | 1226 | 588 | 1224 | 275 | 866 | 13688 |
Dist/m | 7727.72 | 618.03 | 248.40 | 263.17 | 845.15 | 200.14 | 266.27 | 144.13 | 395.66 | 10708.67 |
Ground Truth | RTK/INS | RTK/INS | RTK/INS | RTK/INS | Leica | Leica | Leica | Mocap | Mocap | --- |
![](https://github.com/sjtuyinjie/mypics/raw/main/forgithub/outdoor.png?raw=true)
Figure 7. Outdoor Sequences:all trajectories are mapped in different colors.
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
gate_01 | 2021-07-31 | 16.4g | 172s | dark,around gate | Rosbag | GT |
gate_02 | 2021-07-31 | 27.3g | 327s | dark,loop back | Rosbag | GT |
gate_03 | 2021-08-04 | 21.9g | 283s | day | Rosbag | GT |
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
Circle_01 | 2021-08-03 | 23.3g | 234s | Circle | Rosbag | GT |
Circle_02 | 2021-08-07 | 27.3g | 244s | Circle | Rosbag | GT |
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
street_01 | 2021-08-06 | 75.8g | 1028s | street and buildings,night,zigzag,long-term | Rosbag | GT |
street_02 | 2021-08-03 | 83.2g | 1227s | day,long-term | Rosbag | GT |
street_03 | 2021-08-06 | 21.3g | 354s | night,back and fourth,full speed | Rosbag | GT |
street_04 | 2021-08-03 | 48.7g | 858s | night,around lawn,loop back | Rosbag | GT |
street_05 | 2021-08-04 | 27.4g | 469s | night,staight line | Rosbag | GT |
street_06 | 2021-08-04 | 35.0g | 494s | night,one turn | Rosbag | GT |
street_07 | 2021-08-06 | 77.2g | 929s | dawn,zigzag,sharp turns | Rosbag | GT |
street_08 | 2021-08-06 | 31.2g | 491s | night,loop back,zigzag | Rosbag | GT |
street_09 | 2021-08-07 | 83.2g | 907s | day,zigzag | Rosbag | GT |
street_010 | 2021-08-07 | 86.2g | 910s | day,zigzag | Rosbag | GT |
walk_01 | 2021-08-04 | 21.5g | 291s | day,back and fourth | Rosbag | GT |
![](https://github.com/sjtuyinjie/mypics/raw/main/forgithub/lift.jpg?raw=true)
Figure 8. Lift Sequences:The robot hang around a hall on the first floor and then went to the second floor by lift.A laser scanner track the trajectory outside the lift
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
lift_01 | 2021-08-04 | 18.4g | 225s | lift | Rosbag | GT |
lift_02 | 2021-08-04 | 43.6g | 488s | lift | Rosbag | GT |
lift_03 | 2021-08-15 | 22.3g | 252s | lift | Rosbag | GT |
lift_04 | 2021-08-15 | 27.8g | 299s | lift | Rosbag | GT |
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
hall_01 | 2021-08-01 | 29.1g | 351s | randon walk | Rosbag | GT |
hall_02 | 2021-08-08 | 15.0g | 128s | randon walk | Rosbag | GT |
hall_03 | 2021-08-08 | 20.5g | 164s | randon walk | Rosbag | GT |
hall_04 | 2021-08-15 | 17.7g | 181s | randon walk | Rosbag | GT |
hall_05 | 2021-08-15 | 35.1g | 402s | circle | Rosbag | GT |
![](https://github.com/sjtuyinjie/mypics/raw/main/forgithub/room.png?raw=true)
Figure 9. Room Sequences:under a Motion-capture system with twelve cameras.
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
room_01 | 2021-07-30 | 14.0g | 72s | room,bright | Rosbag | GT |
room_02 | 2021-07-30 | 15.2g | 75s | room,bright | Rosbag | GT |
room_03 | 2021-07-30 | 26.1g | 128s | room,bright | Rosbag | GT |
room_dark_01 | 2021-07-30 | 20.2g | 111s | room,dark | Rosbag | GT |
room_dark_02 | 2021-07-30 | 30.3g | 165s | room,dark | Rosbag | GT |
room_dark_03 | 2021-07-30 | 22.7g | 116s | room,dark | Rosbag | GT |
room_dark_04 | 2021-08-15 | 29.3g | 143s | room,dark | Rosbag | GT |
room_dark_05 | 2021-08-15 | 33.0g | 159s | room,dark | Rosbag | GT |
room_dark_06 | 2021-08-15 | 35.6g | 172s | room,dark | Rosbag | GT |
![](https://github.com/sjtuyinjie/mypics/raw/main/forgithub/door.jpg?raw=true)
Figure 10. Door Sequences:A laser scanner track the robot through a door from indoors to outdoors.
Sequence Name | Collection Date | Total Size | Duration | Features | Rosbag | GT |
---|---|---|---|---|---|---|
door_01 | 2021-08-04 | 35.5g | 461s | outdoor to indoor to outdoor,long-term | Rosbag | GT |
door_02 | 2021-08-04 | 10.5g | 127s | outdoor to indoor,short-term | Rosbag | GT |
For convenience of evaluation, we provide configuration files of some well-known SLAM systems as below:
LINS,
- For rosbag users, first make image view
roscd image_view
rosmake image_view
sudo apt-get install mjpegtools
open a terminal,type roscore.And then open another,type
rosrun image_transport republish compressed in:=/camera/color/image_raw raw out:=/camera/color/image_raw respawn="true"
- For non-rosbag users,just take advantage of following script export_tum,export_euroc and get_csv to get data in formats of Tum or EuRoC.
We use open-source tool evo for evalutation. To install evo,type
pip install evo --upgrade --no-binary evo
To evaluate visual SLAM,type
evo_ape tum street_07.txt your_result.txt -vaps
To evaluate LIDAR SLAM,type
evo_ape tum street_07.txt your_result.txt -vap
To test GNSS based methods,type
evo_ape tum street_07.txt your_result.txt -vp
For camera intrinsics,visit Ocamcalib for omnidirectional model. visit Vins-Fusion for pinhole and MEI model. use Opencv for Kannala Brandt model
For IMU intrinsics,visit Imu_utils
For extrinsics between cameras and IMU,visit Kalibr For extrinsics between Lidar and IMU,visit Lidar_IMU_Calib For extrinsics between cameras and Lidar, visit Autoware
For GNSS based methods like RTKLIB,we usually need to get data in the format of RINEX. To make use of GNSS raw measurements, we use Link toolkit.
We write a ROS driver for UVC cameras to record our thermal-infrared image. UVC ROS driver
This work is licensed under MIT license. International License and is provided for academic purpose. If you are interested in our project for commercial purposes, please contact us on 1195391308@qq.com for further communication.