An error occured when running the visualization script
hbc2020 opened this issue · comments
Dear Author,
Thanks for your great work!
I have followed the guide exactly to prepare the environment and dataset. Then I ran the evaluation script and following results observed,
Command: python tools/test.py projects/configs/VAD/VAD_base_stage_2.py ckpts/VAD_base.pth --launcher none --eval bbox
Results:
-------------- Motion Prediction --------------
EPA_car: 0.6470958226695007
EPA_pedestrian: 0.4735691019078641
ADE_car: 0.686764121055603
ADE_pedestrian: 0.6781615614891052
FDE_car: 0.9205904603004456
FDE_pedestrian: 0.8422962427139282
MR_car: 0.09310622626309127
MR_pedestrian: 0.0728476821192053
-------------- Planning --------------
gt_car:4.503418636452432
gt_pedestrian:2.099042781793319
cnt_ade_car:3.59992185973823
cnt_ade_pedestrian:1.3313147099042781
cnt_fde_car:3.394803672592303
cnt_fde_pedestrian:1.1799179527251416
hit_car:3.078726313733151
hit_pedestrian:1.0939636647782771
fp_car:0.2752490720843915
fp_pedestrian:0.16604805626098848
ADE_car:2.5425851345062256
ADE_pedestrian:0.9197360873222351
FDE_car:3.1252238750457764
FDE_pedestrian:0.9938404560089111
MR_car:0.31607735885915217
MR_pedestrian:0.08595428794686462
plan_L2_1s:0.40841908812844757
plan_L2_2s:0.698031765614144
plan_L2_3s:1.0510727256978822
plan_obj_col_1s:0.0
plan_obj_col_2s:4.883766360617308e-05
plan_obj_col_3s:6.511688674886516e-05
plan_obj_box_col_1s:0.0001953506544246923
plan_obj_box_col_2s:0.0012697792537605002
plan_obj_box_col_3s:0.0034837534119547707
fut_valid_flag:1.0
Formating bboxes of pts_bbox
Start to convert detection format...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6019/6019, 28.2 task/s, elapsed: 214s, ETA: 0s
data/nuscenes/nuscenes_map_anns_val.json exist, not update
Results writes to test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/results_nusc.pkl
Evaluating bboxes of pts_bbox
mAP: 0.3296
mATE: 0.5950
mASE: 0.2843
mAOE: 0.5316
mAVE: 0.4050
mAAE: 0.2333
NDS: 0.4599
Eval time: 62.5s
Per-class results:
Object Class AP ATE ASE AOE AVE AAE
car 0.538 0.389 0.144 0.045 0.361 0.280
truck 0.256 0.616 0.205 0.089 0.375 0.279
bus 0.447 0.537 0.201 0.184 0.528 0.203
trailer 0.031 0.852 0.359 1.232 0.676 0.136
construction_vehicle 0.096 0.935 0.515 1.387 0.092 0.443
pedestrian 0.416 0.547 0.292 0.470 0.349 0.166
motorcycle 0.334 0.530 0.235 0.455 0.652 0.357
bicycle 0.303 0.515 0.275 0.832 0.206 0.003
traffic_cone 0.472 0.473 0.332 nan nan nan
barrier 0.403 0.556 0.285 0.092 nan nan
Formating results & gts by classes
results path: /workspace/projects/VAD/test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/results_nusc.pkl
Formatting ...
Cls data formatting done in 347.788279s!! with /workspace/projects/VAD/test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/cls_formatted.pkl
----------use metric:chamfer----------
----------threshhold:0.5----------
cls:divider done in 23.697950s!!
cls:ped_crossing done in 3.128191s!!
cls:boundary done in 10.045630s!!
+--------------+-------+--------+--------+-------+
| class | gts | dets | recall | ap |
+--------------+-------+--------+--------+-------+
| divider | 27332 | 125728 | 0.525 | 0.287 |
| ped_crossing | 6406 | 49342 | 0.291 | 0.092 |
| boundary | 21050 | 125880 | 0.417 | 0.208 |
+--------------+-------+--------+--------+-------+
| mAP | | | | 0.196 |
+--------------+-------+--------+--------+-------+
----------threshhold:1.0----------
cls:divider done in 23.533183s!!
cls:ped_crossing done in 3.298304s!!
cls:boundary done in 9.815461s!!
+--------------+-------+--------+--------+-------+
| class | gts | dets | recall | ap |
+--------------+-------+--------+--------+-------+
| divider | 27332 | 125728 | 0.767 | 0.559 |
| ped_crossing | 6406 | 49342 | 0.662 | 0.450 |
| boundary | 21050 | 125880 | 0.740 | 0.577 |
+--------------+-------+--------+--------+-------+
| mAP | | | | 0.529 |
+--------------+-------+--------+--------+-------+
----------threshhold:1.5----------
cls:divider done in 24.956200s!!
cls:ped_crossing done in 2.995650s!!
cls:boundary done in 9.802585s!!
+--------------+-------+--------+--------+-------+
| class | gts | dets | recall | ap |
+--------------+-------+--------+--------+-------+
| divider | 27332 | 125728 | 0.866 | 0.699 |
| ped_crossing | 6406 | 49342 | 0.835 | 0.677 |
| boundary | 21050 | 125880 | 0.859 | 0.733 |
+--------------+-------+--------+--------+-------+
| mAP | | | | 0.703 |
+--------------+-------+--------+--------+-------+
divider: 0.5149634877840678
ped_crossing: 0.4063133845726649
boundary: 0.5059408644835154
map: 0.475739245613416
However, when I executed the visulation command, an error occured,
Command: python tools/analysis_tools/visualization.py --result-path test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/results_nusc.pkl --save-path vis_results
Error:
Exception has occurred: ValueError (note: full exception trace is shown but execution is paused at: _run_module_as_main)
all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 3 dimension(s)
File "/workspace/projects/VAD/projects/mmdet3d_plugin/core/bbox/structures/nuscenes_box.py", line 268, in render_fut_trajs_grad_color
fut_coord = np.concatenate((self.center[np.newaxis, :2], fut_coord), axis=0)
File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 567, in visualize_sample
box.render_fut_trajs_grad_color(axes, linewidth=1, mode_idx=mode_idx, fut_ts=6, cmap='autumn')
File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 315, in lidiar_render
visualize_sample(nusc, sample_token, gt_annotations, pred_annotations,
File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 719, in render_sample_data
lidiar_render(sample_toekn, pred_data, out_path=out_path,
File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 747, in
render_sample_data(sample_token_list[id],
File "/opt/conda/envs/vad/lib/python3.8/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/envs/vad/lib/python3.8/runpy.py", line 192, in _run_module_as_main (Current frame)
return _run_code(code, main_globals, None,
ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 3 dimension(s)
Is there any problem with my commands, configurations or some codes need to be updated? Thank you very much!
We haven't encountered this error when performing visualization. It seems that the shape of the two inputs (self.center and fut_coord)is not correct in this line:
fut_coord = np.concatenate((self.center[np.newaxis, :2], fut_coord), axis=0)
the shape of self.center should be [3], and the shape of fut_coord should be [6, 2].
Dear Author, Thanks for your great work! I have followed the guide exactly to prepare the environment and dataset. Then I ran the evaluation script and following results observed, Command: python tools/test.py projects/configs/VAD/VAD_base_stage_2.py ckpts/VAD_base.pth --launcher none --eval bbox Results: -------------- Motion Prediction -------------- EPA_car: 0.6470958226695007 EPA_pedestrian: 0.4735691019078641 ADE_car: 0.686764121055603 ADE_pedestrian: 0.6781615614891052 FDE_car: 0.9205904603004456 FDE_pedestrian: 0.8422962427139282 MR_car: 0.09310622626309127 MR_pedestrian: 0.0728476821192053
-------------- Planning -------------- gt_car:4.503418636452432 gt_pedestrian:2.099042781793319 cnt_ade_car:3.59992185973823 cnt_ade_pedestrian:1.3313147099042781 cnt_fde_car:3.394803672592303 cnt_fde_pedestrian:1.1799179527251416 hit_car:3.078726313733151 hit_pedestrian:1.0939636647782771 fp_car:0.2752490720843915 fp_pedestrian:0.16604805626098848 ADE_car:2.5425851345062256 ADE_pedestrian:0.9197360873222351 FDE_car:3.1252238750457764 FDE_pedestrian:0.9938404560089111 MR_car:0.31607735885915217 MR_pedestrian:0.08595428794686462 plan_L2_1s:0.40841908812844757 plan_L2_2s:0.698031765614144 plan_L2_3s:1.0510727256978822 plan_obj_col_1s:0.0 plan_obj_col_2s:4.883766360617308e-05 plan_obj_col_3s:6.511688674886516e-05 plan_obj_box_col_1s:0.0001953506544246923 plan_obj_box_col_2s:0.0012697792537605002 plan_obj_box_col_3s:0.0034837534119547707 fut_valid_flag:1.0
Formating bboxes of pts_bbox Start to convert detection format... [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6019/6019, 28.2 task/s, elapsed: 214s, ETA: 0s data/nuscenes/nuscenes_map_anns_val.json exist, not update Results writes to test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/results_nusc.pkl Evaluating bboxes of pts_bbox mAP: 0.3296 mATE: 0.5950 mASE: 0.2843 mAOE: 0.5316 mAVE: 0.4050 mAAE: 0.2333 NDS: 0.4599 Eval time: 62.5s
Per-class results: Object Class AP ATE ASE AOE AVE AAE car 0.538 0.389 0.144 0.045 0.361 0.280 truck 0.256 0.616 0.205 0.089 0.375 0.279 bus 0.447 0.537 0.201 0.184 0.528 0.203 trailer 0.031 0.852 0.359 1.232 0.676 0.136 construction_vehicle 0.096 0.935 0.515 1.387 0.092 0.443 pedestrian 0.416 0.547 0.292 0.470 0.349 0.166 motorcycle 0.334 0.530 0.235 0.455 0.652 0.357 bicycle 0.303 0.515 0.275 0.832 0.206 0.003 traffic_cone 0.472 0.473 0.332 nan nan nan barrier 0.403 0.556 0.285 0.092 nan nan Formating results & gts by classes results path: /workspace/projects/VAD/test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/results_nusc.pkl Formatting ... Cls data formatting done in 347.788279s!! with /workspace/projects/VAD/test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/cls_formatted.pkl ----------use metric:chamfer---------- ----------threshhold:0.5---------- cls:divider done in 23.697950s!! cls:ped_crossing done in 3.128191s!! cls:boundary done in 10.045630s!!
+--------------+-------+--------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-------+--------+--------+-------+ | divider | 27332 | 125728 | 0.525 | 0.287 | | ped_crossing | 6406 | 49342 | 0.291 | 0.092 | | boundary | 21050 | 125880 | 0.417 | 0.208 | +--------------+-------+--------+--------+-------+ | mAP | | | | 0.196 | +--------------+-------+--------+--------+-------+ ----------threshhold:1.0---------- cls:divider done in 23.533183s!! cls:ped_crossing done in 3.298304s!! cls:boundary done in 9.815461s!!
+--------------+-------+--------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-------+--------+--------+-------+ | divider | 27332 | 125728 | 0.767 | 0.559 | | ped_crossing | 6406 | 49342 | 0.662 | 0.450 | | boundary | 21050 | 125880 | 0.740 | 0.577 | +--------------+-------+--------+--------+-------+ | mAP | | | | 0.529 | +--------------+-------+--------+--------+-------+ ----------threshhold:1.5---------- cls:divider done in 24.956200s!! cls:ped_crossing done in 2.995650s!! cls:boundary done in 9.802585s!!
+--------------+-------+--------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-------+--------+--------+-------+ | divider | 27332 | 125728 | 0.866 | 0.699 | | ped_crossing | 6406 | 49342 | 0.835 | 0.677 | | boundary | 21050 | 125880 | 0.859 | 0.733 | +--------------+-------+--------+--------+-------+ | mAP | | | | 0.703 | +--------------+-------+--------+--------+-------+ divider: 0.5149634877840678 ped_crossing: 0.4063133845726649 boundary: 0.5059408644835154 map: 0.475739245613416
However, when I executed the visulation command, an error occured, Command: python tools/analysis_tools/visualization.py --result-path test/VAD_base_stage_2/Sat_Sep__9_02_04_54_2023/pts_bbox/results_nusc.pkl --save-path vis_results Error: Exception has occurred: ValueError (note: full exception trace is shown but execution is paused at: _run_module_as_main) all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 3 dimension(s) File "/workspace/projects/VAD/projects/mmdet3d_plugin/core/bbox/structures/nuscenes_box.py", line 268, in render_fut_trajs_grad_color fut_coord = np.concatenate((self.center[np.newaxis, :2], fut_coord), axis=0) File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 567, in visualize_sample box.render_fut_trajs_grad_color(axes, linewidth=1, mode_idx=mode_idx, fut_ts=6, cmap='autumn') File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 315, in lidiar_render visualize_sample(nusc, sample_token, gt_annotations, pred_annotations, File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 719, in render_sample_data lidiar_render(sample_toekn, pred_data, out_path=out_path, File "/workspace/projects/VAD/tools/analysis_tools/visualization.py", line 747, in render_sample_data(sample_token_list[id], File "/opt/conda/envs/vad/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/opt/conda/envs/vad/lib/python3.8/runpy.py", line 192, in _run_module_as_main (Current frame) return _run_code(code, main_globals, None, ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 1 has 3 dimension(s) Is there any problem with my commands, configurations or some codes need to be updated? Thank you very much!
I also encountered the same problem. Have you solved it?
My comment might be later and would like to share to others who encounter the same issue. The solution to address this bug is to navigate to the projects/mmdet3d_plugin/core/bbox/structures/nuscenes_box.py and modify the line:
fut_coords = fut_coords[[mode_idx]]
to
fut_coords = fut_coords[[mode_idx]][0]