confusing about the precision calculation in many_to_many method Deteval.py
fdengmark opened this issue · comments
Hi I'm reading the Deteval.py script and I'm confusing about the precision calculation in the many_to_many() method. (line 203)
when you calculating recall, you considered the num_qualified_sigma_candidates, but you don't consider num_qualified_tau_candidates when you calculate precision. Moreover, given the following condition (line 199), I think this method really should be called as many_to_one instead of many_to_many.
In summary, I think if you don't consider num_qualified_tau_candidates when you calculate precision in many_to_many method, and you only check if np.sum(local_tau_table[qualified_sigma_candidates, det_id]) >= tp. This method really should be called as many_to_one, and you probably need another many_to_many method.
Regardless of this issue, I really appreciate your work and I learned a lot from your code. Thank you.
Hi there, we noticed this issue 2 weeks ago and patched it in our latest commit. Thanks for pointing it out!
Yes we did not consider the many-to-many case in the script, as the implementation is more complicated than the other cases and it is relatively rare in compared to other cases. However, we do plan to implement it in the near future. We would appreciate if you'd like to contribute.
Thank you for your kind words. =)
Sorry for the late reply, I’m glad if I can contribute to this project. I’m actually very new to github, so I don’t really know what is the contribution process. I will finish the many to many part before next week. Moreover, I have coded the precision recall computation as an method so you can call it with other script. (at least I think it is more convenient to use)