The released checkpoint and result file cannot reach the released mAP value

[su]

Hi, I have evaluate the released flow checkpoint and flow result following the default setting, both reached a lower mAP than the release 47.42@.5tIou on testset: 0.4683 and 0.4662. The approx. 1 percent difference seems nontrivial.

Besides, I have also tried to train it for a few times and eval different checkpoints, which all yields ~1% gap with 47.42% despite a jittering around .2% each training.

Please double-check the released ckpts and results, and hopefully point out some mistasks that I may have made. Appreciate that!

[Runhao Zeng]

Hi, I have tested the flow checkpoint on our server and obtained the following results:

Note that the mAP will change slightly when you test the same model multiple times. The reason is that we adopt GraphSage to sample the neighborhoods randomly (i.e., sampling #child_num nodes from 10 neighborhoods). To test the model without sampling, you could set the #child_num to 10 and #adj_num to 111 in "data/dataset_cfg.yaml".