Mask module bootstrap does not converge
fengziyue opened this issue · comments
Hi:
Thank you for sharing this amazing work!
I tried to train the model several times but the mask module bootstrapping can not converge on the KITTI dataset. The only thing I changed in the code is from here .
The training iou is increasing but the validation iou is decreasing so is it overfitting?
The result looks very noisy even in the training set:
However, the mask module refinement converged as well as claimed in the paper.
below is the training log from mask module bootstrapping:
2022-05-21 18:56:28,980 - train - INFO - 1503377 trainable parameters
2022-05-21 19:33:24,088 - trainer - INFO - epoch : 1
2022-05-21 19:33:24,089 - trainer - INFO - loss : 0.33975716474378714
2022-05-21 19:33:24,090 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 19:33:24,091 - trainer - INFO - abs_rel_sparse_metric: 1.975
2022-05-21 19:33:24,092 - trainer - INFO - rmse_sparse_metric: 1.975
2022-05-21 19:33:24,092 - trainer - INFO - loss_iou : 0.08929756164550781
2022-05-21 19:33:24,093 - trainer - INFO - loss_prec : 0.09143932342529297
2022-05-21 19:33:24,094 - trainer - INFO - loss_loss : 0.33975664774576825
2022-05-21 19:33:24,095 - trainer - INFO - loss_acc : 0.9185029093424479
2022-05-21 19:33:24,096 - trainer - INFO - loss_rec : 0.9724029541015625
2022-05-21 19:33:24,096 - trainer - INFO - val_loss : 2.165919481880135
2022-05-21 19:33:24,097 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 19:33:24,098 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 19:33:24,099 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 19:33:24,100 - trainer - INFO - val_loss_iou : 0.09720667865541247
2022-05-21 19:33:24,100 - trainer - INFO - val_loss_prec : 0.10425782865948147
2022-05-21 19:33:24,101 - trainer - INFO - val_loss_loss : 2.165919621785482
2022-05-21 19:33:24,102 - trainer - INFO - val_loss_acc : 0.8980947070651584
2022-05-21 19:33:24,103 - trainer - INFO - val_loss_rec : 0.7917230923970541
2022-05-21 19:33:24,324 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 19:33:24,545 - trainer - INFO - Saving current best: model_best.pth ...
2022-05-21 20:09:51,592 - trainer - INFO - epoch : 2
2022-05-21 20:09:51,593 - trainer - INFO - loss : 0.18604853542870842
2022-05-21 20:09:51,596 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 20:09:51,599 - trainer - INFO - abs_rel_sparse_metric: 1.9338541666666667
2022-05-21 20:09:51,601 - trainer - INFO - rmse_sparse_metric: 1.9338541666666667
2022-05-21 20:09:51,604 - trainer - INFO - loss_iou : 0.13620360056559244
2022-05-21 20:09:51,606 - trainer - INFO - loss_prec : 0.13805244445800782
2022-05-21 20:09:51,609 - trainer - INFO - loss_loss : 0.1860488001505534
2022-05-21 20:09:51,612 - trainer - INFO - loss_acc : 0.9541281127929687
2022-05-21 20:09:51,614 - trainer - INFO - loss_rec : 0.9810747273763021
2022-05-21 20:09:51,617 - trainer - INFO - val_loss : 3.564138030219409
2022-05-21 20:09:51,619 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 20:09:51,621 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 20:09:51,623 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 20:09:51,625 - trainer - INFO - val_loss_iou : 0.10033636622958714
2022-05-21 20:09:51,627 - trainer - INFO - val_loss_prec : 0.1108185847600301
2022-05-21 20:09:51,629 - trainer - INFO - val_loss_loss : 3.5641377766927085
2022-05-21 20:09:51,631 - trainer - INFO - val_loss_acc : 0.9085981580946181
2022-05-21 20:09:51,633 - trainer - INFO - val_loss_rec : 0.7776173485649956
2022-05-21 20:09:52,095 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 20:46:30,289 - trainer - INFO - epoch : 3
2022-05-21 20:46:30,292 - trainer - INFO - loss : 0.13589516458225262
2022-05-21 20:46:30,298 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 20:46:30,303 - trainer - INFO - abs_rel_sparse_metric: 2.1313541666666667
2022-05-21 20:46:30,308 - trainer - INFO - rmse_sparse_metric: 2.1313541666666667
2022-05-21 20:46:30,313 - trainer - INFO - loss_iou : 0.17558348337809246
2022-05-21 20:46:30,318 - trainer - INFO - loss_prec : 0.17756632486979165
2022-05-21 20:46:30,323 - trainer - INFO - loss_loss : 0.1358951695760091
2022-05-21 20:46:30,329 - trainer - INFO - loss_acc : 0.9673453776041666
2022-05-21 20:46:30,332 - trainer - INFO - loss_rec : 0.9840897623697916
2022-05-21 20:46:30,337 - trainer - INFO - val_loss : 2.532514634024766
2022-05-21 20:46:30,342 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 20:46:30,347 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 20:46:30,353 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 20:46:30,358 - trainer - INFO - val_loss_iou : 0.07900547981262207
2022-05-21 20:46:30,364 - trainer - INFO - val_loss_prec : 0.08653933472103542
2022-05-21 20:46:30,371 - trainer - INFO - val_loss_loss : 2.532514360215929
2022-05-21 20:46:30,377 - trainer - INFO - val_loss_acc : 0.8775181240505643
2022-05-21 20:46:30,382 - trainer - INFO - val_loss_rec : 0.7745491663614908
2022-05-21 20:46:30,914 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 21:22:54,544 - trainer - INFO - epoch : 4
2022-05-21 21:22:54,551 - trainer - INFO - loss : 0.11389048684442969
2022-05-21 21:22:54,571 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 21:22:54,589 - trainer - INFO - abs_rel_sparse_metric: 2.0490625
2022-05-21 21:22:54,608 - trainer - INFO - rmse_sparse_metric: 2.0490625
2022-05-21 21:22:54,621 - trainer - INFO - loss_iou : 0.1996185811360677
2022-05-21 21:22:54,638 - trainer - INFO - loss_prec : 0.20159257253011068
2022-05-21 21:22:54,657 - trainer - INFO - loss_loss : 0.1138906987508138
2022-05-21 21:22:54,675 - trainer - INFO - loss_acc : 0.9730851236979167
2022-05-21 21:22:54,692 - trainer - INFO - loss_rec : 0.9856961059570313
2022-05-21 21:22:54,709 - trainer - INFO - val_loss : 8.58495722938743
2022-05-21 21:22:54,726 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 21:22:54,741 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 21:22:54,756 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 21:22:54,772 - trainer - INFO - val_loss_iou : 0.10192386971579658
2022-05-21 21:22:54,786 - trainer - INFO - val_loss_prec : 0.19616170724232992
2022-05-21 21:22:54,802 - trainer - INFO - val_loss_loss : 8.584956698947483
2022-05-21 21:22:54,818 - trainer - INFO - val_loss_acc : 0.9627514945136176
2022-05-21 21:22:54,834 - trainer - INFO - val_loss_rec : 0.3266310956743028
2022-05-21 21:22:55,087 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 21:59:20,427 - trainer - INFO - epoch : 5
2022-05-21 21:59:20,429 - trainer - INFO - loss : 0.09532881293710185
2022-05-21 21:59:20,430 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 21:59:20,431 - trainer - INFO - abs_rel_sparse_metric: 2.378229166666667
2022-05-21 21:59:20,432 - trainer - INFO - rmse_sparse_metric: 2.378229166666667
2022-05-21 21:59:20,432 - trainer - INFO - loss_iou : 0.22963691711425782
2022-05-21 21:59:20,433 - trainer - INFO - loss_prec : 0.23156720479329426
2022-05-21 21:59:20,434 - trainer - INFO - loss_loss : 0.09532897313435873
2022-05-21 21:59:20,435 - trainer - INFO - loss_acc : 0.9782980346679687
2022-05-21 21:59:20,436 - trainer - INFO - loss_rec : 0.9877736409505208
2022-05-21 21:59:20,437 - trainer - INFO - val_loss : 10.559455184162491
2022-05-21 21:59:20,437 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 21:59:20,438 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 21:59:20,439 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 21:59:20,440 - trainer - INFO - val_loss_iou : 0.08753267923990886
2022-05-21 21:59:20,441 - trainer - INFO - val_loss_prec : 0.1575686534245809
2022-05-21 21:59:20,441 - trainer - INFO - val_loss_loss : 10.559455023871529
2022-05-21 21:59:20,442 - trainer - INFO - val_loss_acc : 0.9632636176215278
2022-05-21 21:59:20,443 - trainer - INFO - val_loss_rec : 0.26647456487019855
2022-05-21 21:59:20,643 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 22:35:45,561 - trainer - INFO - epoch : 6
2022-05-21 22:35:45,562 - trainer - INFO - loss : 0.07562574363061988
2022-05-21 22:35:45,563 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 22:35:45,564 - trainer - INFO - abs_rel_sparse_metric: 2.0819791666666667
2022-05-21 22:35:45,565 - trainer - INFO - rmse_sparse_metric: 2.0819791666666667
2022-05-21 22:35:45,566 - trainer - INFO - loss_iou : 0.2568237050374349
2022-05-21 22:35:45,567 - trainer - INFO - loss_prec : 0.25863550821940107
2022-05-21 22:35:45,567 - trainer - INFO - loss_loss : 0.0756257692972819
2022-05-21 22:35:45,568 - trainer - INFO - loss_acc : 0.9828921508789062
2022-05-21 22:35:45,569 - trainer - INFO - loss_rec : 0.9893025716145833
2022-05-21 22:35:45,570 - trainer - INFO - val_loss : 11.668905624705884
2022-05-21 22:35:45,571 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 22:35:45,572 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 22:35:45,573 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 22:35:45,573 - trainer - INFO - val_loss_iou : 0.08269011974334717
2022-05-21 22:35:45,574 - trainer - INFO - val_loss_prec : 0.19104923142327201
2022-05-21 22:35:45,575 - trainer - INFO - val_loss_loss : 11.668904622395834
2022-05-21 22:35:45,576 - trainer - INFO - val_loss_acc : 0.9682659573025174
2022-05-21 22:35:45,577 - trainer - INFO - val_loss_rec : 0.18886461522844103
2022-05-21 22:35:45,747 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 23:12:12,250 - trainer - INFO - epoch : 7
2022-05-21 23:12:12,253 - trainer - INFO - loss : 0.06769100399202822
2022-05-21 23:12:12,259 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 23:12:12,266 - trainer - INFO - abs_rel_sparse_metric: 2.3123958333333334
2022-05-21 23:12:12,272 - trainer - INFO - rmse_sparse_metric: 2.3123958333333334
2022-05-21 23:12:12,278 - trainer - INFO - loss_iou : 0.27149088541666666
2022-05-21 23:12:12,283 - trainer - INFO - loss_prec : 0.27367930094401044
2022-05-21 23:12:12,288 - trainer - INFO - loss_loss : 0.0676910400390625
2022-05-21 23:12:12,293 - trainer - INFO - loss_acc : 0.9849922688802083
2022-05-21 23:12:12,297 - trainer - INFO - loss_rec : 0.9895144653320312
2022-05-21 23:12:12,300 - trainer - INFO - val_loss : 12.180131924545599
2022-05-21 23:12:12,304 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 23:12:12,308 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 23:12:12,312 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 23:12:12,316 - trainer - INFO - val_loss_iou : 0.08412468433380127
2022-05-21 23:12:12,320 - trainer - INFO - val_loss_prec : 0.15997138288285997
2022-05-21 23:12:12,325 - trainer - INFO - val_loss_loss : 12.180132548014322
2022-05-21 23:12:12,328 - trainer - INFO - val_loss_acc : 0.9604445563422309
2022-05-21 23:12:12,331 - trainer - INFO - val_loss_rec : 0.2399449348449707
2022-05-21 23:12:12,700 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-21 23:48:32,490 - trainer - INFO - epoch : 8
2022-05-21 23:48:32,503 - trainer - INFO - loss : 0.06005031946408053
2022-05-21 23:48:32,553 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-21 23:48:32,580 - trainer - INFO - abs_rel_sparse_metric: 2.2465625
2022-05-21 23:48:32,610 - trainer - INFO - rmse_sparse_metric: 2.2465625
2022-05-21 23:48:32,640 - trainer - INFO - loss_iou : 0.2933336893717448
2022-05-21 23:48:32,670 - trainer - INFO - loss_prec : 0.29533246358235676
2022-05-21 23:48:32,701 - trainer - INFO - loss_loss : 0.06005029678344727
2022-05-21 23:48:32,733 - trainer - INFO - loss_acc : 0.9866059366861979
2022-05-21 23:48:32,764 - trainer - INFO - loss_rec : 0.9910132853190105
2022-05-21 23:48:32,795 - trainer - INFO - val_loss : 14.821350382227036
2022-05-21 23:48:32,825 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-21 23:48:32,855 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-21 23:48:32,866 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-21 23:48:32,870 - trainer - INFO - val_loss_iou : 0.08032339811325073
2022-05-21 23:48:32,885 - trainer - INFO - val_loss_prec : 0.2632046275668674
2022-05-21 23:48:32,913 - trainer - INFO - val_loss_loss : 14.82135009765625
2022-05-21 23:48:32,939 - trainer - INFO - val_loss_acc : 0.9833286073472765
2022-05-21 23:48:32,964 - trainer - INFO - val_loss_rec : 0.14326214790344238
2022-05-21 23:48:33,342 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 00:24:59,842 - trainer - INFO - epoch : 9
2022-05-22 00:24:59,843 - trainer - INFO - loss : 0.058349815968667826
2022-05-22 00:24:59,844 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 00:24:59,845 - trainer - INFO - abs_rel_sparse_metric: 2.221875
2022-05-22 00:24:59,846 - trainer - INFO - rmse_sparse_metric: 2.221875
2022-05-22 00:24:59,847 - trainer - INFO - loss_iou : 0.3101000722249349
2022-05-22 00:24:59,847 - trainer - INFO - loss_prec : 0.3118464660644531
2022-05-22 00:24:59,848 - trainer - INFO - loss_loss : 0.058349742889404296
2022-05-22 00:24:59,849 - trainer - INFO - loss_acc : 0.9872409057617187
2022-05-22 00:24:59,850 - trainer - INFO - loss_rec : 0.9917488606770833
2022-05-22 00:24:59,851 - trainer - INFO - val_loss : 12.517815000481075
2022-05-22 00:24:59,851 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 00:24:59,852 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 00:24:59,853 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 00:24:59,854 - trainer - INFO - val_loss_iou : 0.07787870698504978
2022-05-22 00:24:59,855 - trainer - INFO - val_loss_prec : 0.2857825491163466
2022-05-22 00:24:59,856 - trainer - INFO - val_loss_loss : 12.517814636230469
2022-05-22 00:24:59,856 - trainer - INFO - val_loss_acc : 0.9768172370062934
2022-05-22 00:24:59,857 - trainer - INFO - val_loss_rec : 0.13944166236453587
2022-05-22 00:25:00,027 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 01:01:40,023 - trainer - INFO - epoch : 10
2022-05-22 01:01:40,027 - trainer - INFO - loss : 0.0482115954470828
2022-05-22 01:01:40,037 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 01:01:40,048 - trainer - INFO - abs_rel_sparse_metric: 1.9503125
2022-05-22 01:01:40,059 - trainer - INFO - rmse_sparse_metric: 1.9503125
2022-05-22 01:01:40,069 - trainer - INFO - loss_iou : 0.3312089538574219
2022-05-22 01:01:40,078 - trainer - INFO - loss_prec : 0.3328692626953125
2022-05-22 01:01:40,087 - trainer - INFO - loss_loss : 0.04821149190266927
2022-05-22 01:01:40,096 - trainer - INFO - loss_acc : 0.9895101928710938
2022-05-22 01:01:40,105 - trainer - INFO - loss_rec : 0.9922865804036458
2022-05-22 01:01:40,113 - trainer - INFO - val_loss : 13.955339850650894
2022-05-22 01:01:40,122 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 01:01:40,131 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 01:01:40,141 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 01:01:40,150 - trainer - INFO - val_loss_iou : 0.0780107511414422
2022-05-22 01:01:40,159 - trainer - INFO - val_loss_prec : 0.26500195927090114
2022-05-22 01:01:40,168 - trainer - INFO - val_loss_loss : 13.95534176296658
2022-05-22 01:01:40,176 - trainer - INFO - val_loss_acc : 0.9831354353162978
2022-05-22 01:01:40,184 - trainer - INFO - val_loss_rec : 0.13859159416622585
2022-05-22 01:01:40,399 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 01:38:00,857 - trainer - INFO - epoch : 11
2022-05-22 01:38:00,858 - trainer - INFO - loss : 0.04933202152824833
2022-05-22 01:38:00,859 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 01:38:00,859 - trainer - INFO - abs_rel_sparse_metric: 2.0902083333333334
2022-05-22 01:38:00,860 - trainer - INFO - rmse_sparse_metric: 2.0902083333333334
2022-05-22 01:38:00,861 - trainer - INFO - loss_iou : 0.3328869883219401
2022-05-22 01:38:00,862 - trainer - INFO - loss_prec : 0.33459765116373696
2022-05-22 01:38:00,863 - trainer - INFO - loss_loss : 0.04933205286661784
2022-05-22 01:38:00,864 - trainer - INFO - loss_acc : 0.9893855794270834
2022-05-22 01:38:00,865 - trainer - INFO - loss_rec : 0.9925393676757812
2022-05-22 01:38:00,866 - trainer - INFO - val_loss : 16.757661278049152
2022-05-22 01:38:00,867 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 01:38:00,868 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 01:38:00,869 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 01:38:00,870 - trainer - INFO - val_loss_iou : 0.06531501478619045
2022-05-22 01:38:00,871 - trainer - INFO - val_loss_prec : 0.3991677496168349
2022-05-22 01:38:00,871 - trainer - INFO - val_loss_loss : 16.75765821668837
2022-05-22 01:38:00,872 - trainer - INFO - val_loss_acc : 0.9846226374308268
2022-05-22 01:38:00,873 - trainer - INFO - val_loss_rec : 0.10530128081639607
2022-05-22 01:38:01,035 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 02:14:19,960 - trainer - INFO - epoch : 12
2022-05-22 02:14:19,968 - trainer - INFO - loss : 0.04878552877263549
2022-05-22 02:14:19,987 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 02:14:19,999 - trainer - INFO - abs_rel_sparse_metric: 2.0572916666666665
2022-05-22 02:14:20,012 - trainer - INFO - rmse_sparse_metric: 2.0572916666666665
2022-05-22 02:14:20,025 - trainer - INFO - loss_iou : 0.34831891377766927
2022-05-22 02:14:20,042 - trainer - INFO - loss_prec : 0.35023468017578124
2022-05-22 02:14:20,057 - trainer - INFO - loss_loss : 0.04878554026285807
2022-05-22 02:14:20,073 - trainer - INFO - loss_acc : 0.9892347208658854
2022-05-22 02:14:20,087 - trainer - INFO - loss_rec : 0.9926947021484375
2022-05-22 02:14:20,101 - trainer - INFO - val_loss : 14.85178346435229
2022-05-22 02:14:20,116 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 02:14:20,129 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 02:14:20,228 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 02:14:20,245 - trainer - INFO - val_loss_iou : 0.07334701220194499
2022-05-22 02:14:20,266 - trainer - INFO - val_loss_prec : 0.3393416934543186
2022-05-22 02:14:20,287 - trainer - INFO - val_loss_loss : 14.851782904730904
2022-05-22 02:14:20,308 - trainer - INFO - val_loss_acc : 0.9831968943277994
2022-05-22 02:14:20,331 - trainer - INFO - val_loss_rec : 0.12560137112935385
2022-05-22 02:14:20,523 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 02:50:43,264 - trainer - INFO - epoch : 13
2022-05-22 02:50:43,266 - trainer - INFO - loss : 0.04147944352905446
2022-05-22 02:50:43,267 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 02:50:43,269 - trainer - INFO - abs_rel_sparse_metric: 1.975
2022-05-22 02:50:43,270 - trainer - INFO - rmse_sparse_metric: 1.975
2022-05-22 02:50:43,272 - trainer - INFO - loss_iou : 0.3653294626871745
2022-05-22 02:50:43,273 - trainer - INFO - loss_prec : 0.3668243408203125
2022-05-22 02:50:43,273 - trainer - INFO - loss_loss : 0.04147947947184245
2022-05-22 02:50:43,274 - trainer - INFO - loss_acc : 0.9910907999674479
2022-05-22 02:50:43,275 - trainer - INFO - loss_rec : 0.9932733154296876
2022-05-22 02:50:43,276 - trainer - INFO - val_loss : 10.780205345609122
2022-05-22 02:50:43,277 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 02:50:43,278 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 02:50:43,279 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 02:50:43,280 - trainer - INFO - val_loss_iou : 0.08617289861043294
2022-05-22 02:50:43,281 - trainer - INFO - val_loss_prec : 0.17777089277903238
2022-05-22 02:50:43,282 - trainer - INFO - val_loss_loss : 10.780204772949219
2022-05-22 02:50:43,283 - trainer - INFO - val_loss_acc : 0.9674543804592557
2022-05-22 02:50:43,284 - trainer - INFO - val_loss_rec : 0.2760783831278483
2022-05-22 02:50:43,848 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 03:27:08,776 - trainer - INFO - epoch : 14
2022-05-22 03:27:08,777 - trainer - INFO - loss : 0.03722713669809006
2022-05-22 03:27:08,779 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 03:27:08,780 - trainer - INFO - abs_rel_sparse_metric: 1.8515625
2022-05-22 03:27:08,782 - trainer - INFO - rmse_sparse_metric: 1.8515625
2022-05-22 03:27:08,784 - trainer - INFO - loss_iou : 0.3863360850016276
2022-05-22 03:27:08,785 - trainer - INFO - loss_prec : 0.38789883931477864
2022-05-22 03:27:08,786 - trainer - INFO - loss_loss : 0.0372270933787028
2022-05-22 03:27:08,788 - trainer - INFO - loss_acc : 0.9919660441080729
2022-05-22 03:27:08,789 - trainer - INFO - loss_rec : 0.9940956624348959
2022-05-22 03:27:08,790 - trainer - INFO - val_loss : 15.135421420551008
2022-05-22 03:27:08,791 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 03:27:08,792 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 03:27:08,793 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 03:27:08,794 - trainer - INFO - val_loss_iou : 0.08546278874079387
2022-05-22 03:27:08,795 - trainer - INFO - val_loss_prec : 0.3615957631005181
2022-05-22 03:27:08,796 - trainer - INFO - val_loss_loss : 15.13542005750868
2022-05-22 03:27:08,797 - trainer - INFO - val_loss_acc : 0.9845504760742188
2022-05-22 03:27:08,798 - trainer - INFO - val_loss_rec : 0.13796117570665148
2022-05-22 03:27:09,118 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...
2022-05-22 04:03:31,666 - trainer - INFO - epoch : 15
2022-05-22 04:03:31,672 - trainer - INFO - loss : 0.034510827331791914
2022-05-22 04:03:31,679 - trainer - INFO - a1_sparse_metric: 0.0
2022-05-22 04:03:31,687 - trainer - INFO - abs_rel_sparse_metric: 1.9503125
2022-05-22 04:03:31,695 - trainer - INFO - rmse_sparse_metric: 1.9503125
2022-05-22 04:03:31,703 - trainer - INFO - loss_iou : 0.3962159729003906
2022-05-22 04:03:31,711 - trainer - INFO - loss_prec : 0.3980547078450521
2022-05-22 04:03:31,720 - trainer - INFO - loss_loss : 0.03451083501180013
2022-05-22 04:03:31,729 - trainer - INFO - loss_acc : 0.9925803629557292
2022-05-22 04:03:31,739 - trainer - INFO - loss_rec : 0.9941372680664062
2022-05-22 04:03:31,749 - trainer - INFO - val_loss : 16.985714921520817
2022-05-22 04:03:31,754 - trainer - INFO - val_a1_sparse_metric: 0.0
2022-05-22 04:03:31,765 - trainer - INFO - val_abs_rel_sparse_metric: 79.0
2022-05-22 04:03:31,774 - trainer - INFO - val_rmse_sparse_metric: 79.0
2022-05-22 04:03:31,784 - trainer - INFO - val_loss_iou : 0.0905357797940572
2022-05-22 04:03:31,795 - trainer - INFO - val_loss_prec : 0.2897793451944987
2022-05-22 04:03:31,805 - trainer - INFO - val_loss_loss : 16.985714382595486
2022-05-22 04:03:31,815 - trainer - INFO - val_loss_acc : 0.9830976062350802
2022-05-22 04:03:31,826 - trainer - INFO - val_loss_rec : 0.16758330663045248
2022-05-22 04:03:32,004 - trainer - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...