Evaluation

[langelesl]

Hi.

Thanks for sharing your code. After the command " $ rostopic echo /laser_odom_to_init/pose/pose/position > 00_result.txt " I got a result.txt like this:

And I tried to use benchmarking.m in your repo to evaluate the result with the 00.txt of kitti ground truth files but failed. Because the programm could not find the string "position" in result.txt as the sign of coordinates for a frame. Actually it seems like the benchmarking.m expected a result.txt with both position and orientation like this:

So could you please tell me what modification I should make to output a "right" result.txt as expected?

Thanks.

[langelesl]

When the command was changed to "$ rostopic echo /laser_odom_to_init/pose/pose/position > 00_result.txt" , the position and orientation could be outputted in result.txt. I just used benchmarking.m, the result of evaluation looked weird. The z-axis is reversed and x,y are not so accurate.

May I get some advices of tuning from you?

Thanks.

[Steven Liu]

Hello, I believe we had hard-coded the rotation and translation offsets in that benchmarking script for sequence 08 in the odometry dataset. This looks like a different sequence, so look at the initial translation and rotation offsets between the results and ground truth to figure out what your parameters might be.

The z-axis problem seems like an initialization problem actually. But you could still apply post processing to reverse the z-axis. Double check the data flow of your ROS nodes too, sometimes information packets are dropped if running at full speed.

[langelesl]

Thanks for your reply, Steven. What I am using is sequence 00. I have found the parts you mentioned in the benchmarking script. I have 2 questions.

1. Seems like you used Xoffset, Zoffset and a rotation matrix R to correct the initial offsets between result and ground truth. I think it could help to move and rotate the result trajectory to the "right" place to match the ground truth trajectory better. Could you tell me how you get the theta to calculate the matrix R?

2. Even if I fixed the initial offsets and Z-axis, I'm afraid that the result trajectory of sequence 00 I got still seems to be far away from the ground truth. I ran the rosbag with 0.1 speed. Did you use synchronized raw data (IMU: 10Hz) or unsynchronized raw data (IMU: 100Hz) of KITTI sequence 08?

[Yuanwen Yue]

@langelesl Hello, I met the same question with you. I evaluate the result with the result.txt of kitti ground truth files but failed. The program could not find the string "position" in result.txt. Could you tell me how you modified the command to get the right result.txt. Thank you!

[langelesl]

@langelesl Hello, I met the same question with you. I evaluate the result with the result.txt of kitti ground truth files but failed. The program could not find the string "position" in result.txt. Could you tell me how you modified the command to get the right result.txt. Thank you!

Sure. I used "$rostopic echo /laser_odom_to_init/pose/pose > 00_result.txt" instead of "$rostopic echo /laser_odom_to_init/pose/pose/position > 00_result.txt". Then both position and orientation informations would be saved in the output file with titles "position" and "orientation".

[Yuanwen Yue]

@langelesl Thanks for your reply. I test sequence 06 and the result of evaluation looked weird. I used synchronized raw data and ran the rosbag with full speed. I saw your result of evaluation of sequence 08 and I found that both results rotated clockwise relative to the ground truth. I wonder if you have solved this problem. Did you use evaluate_odometry.cpp provided in the odometry development kit?
Thanks.

[langelesl]

@langelesl Thanks for your reply. I test sequence 06 and the result of evaluation looked weird. I used synchronized raw data and ran the rosbag with full speed. I saw your result of evaluation of sequence 08 and I found that both results rotated clockwise relative to the ground truth. I wonder if you have solved this problem. Did you use evaluate_odometry.cpp provided in the odometry development kit?
Thanks.

No, I used this repo to evaluate the result. Maybe you could try recording "/aft_mapped_to_init/pose/pose" as result. I guess the trajectory could look better after map optimization.

[Yuanwen Yue]

@langelesl Thanks for your reply. I test sequence 06 and the result of evaluation looked weird. I used synchronized raw data and ran the rosbag with full speed. I saw your result of evaluation of sequence 08 and I found that both results rotated clockwise relative to the ground truth. I wonder if you have solved this problem. Did you use evaluate_odometry.cpp provided in the odometry development kit?
Thanks.

No, I used this repo to evaluate the result. Maybe you could try recording "/aft_mapped_to_init/pose/pose" as result. I guess the trajectory could look better after map optimization.

Thanks for your reply. I also try to use this repo to evaluate the result. One problem is that it requires that 'KITTI pose files must have 12 entries per row and no trailing delimiter at the end of the rows (space)'. Could you tell me what command to use to record the right result file?

[langelesl]

@ywyue Sorry for the late reply, have you solved the problem? Yes it is easy to stuck in because of the KITTI-format. But I think it should be easy to find the solutions with the hints from the issues there. I closed this issue, feel free to send me a E-Mail if you need any help. [e-mail]