Giters

MIC-DKFZ / nnDetection

nnDetection is a self-configuring framework for 3D (volumetric) medical object detection which can be applied to new data sets without manual intervention. It includes guides for 12 data sets that were used to develop and evaluate the performance of the proposed method.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Question] The accuracy issue in reproducing the nndetection algorithm

pengwuke opened this issue · comments

commented

Thank you very much for your work and contribution. We encountered issues when reproducing the nndetection algorithm on the Luna16 dataset. Initially, we tested the results on one fold and observed differences in accuracy compared to our expectations. Subsequently, we ran the algorithm on all folds and performed ensemble, but the results remained consistent. Could we have missed any steps leading to such outcomes? Any suggestions would be greatly appreciated.
20231221104716

commented

Dear @pengwuke ,

I guess your are referring to reproducing the numbers which are in the paper.

The plots you show here are produced by nndet_eval which (1) is based on bounding box IoU (2) does not include ignore locations. To get LUNA complient results you need to use the official LUNA evaluation script (as noted in the project readme of nnDet ;) ) which is provided by the challenge organisers, that will use (1) a center point based criterion (center point in radius of lesion) (2) includes a huge list of locations which are not counted as false positive predictions (please refer to their publication for more information on this). Especially (2) will boost the performance for low number of false positives and will allow for reproducing the numbers from our paper.

Best,
Michael

commented

This issue is stale because it has been open for 30 days with no activity.

commented

This issue was closed because it has been inactive for 14 days since being marked as stale.