Giters

MIC-DKFZ / nnDetection

nnDetection is a self-configuring framework for 3D (volumetric) medical object detection which can be applied to new data sets without manual intervention. It includes guides for 12 data sets that were used to develop and evaluate the performance of the proposed method.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Question] Defining test_labels and target_class for custom dataser

salusanga opened this issue · comments

commented

Hi,
I have a custom dataset for brain metastasis detection. It is a binary task, in the original dataset "1" is the FG and "0" is BG.
I brought it to the nnDetection format, with the instances in the masks and the new FG class being "0".

In this context, how should I set "test_labels" and "target_class"?

Thanks a lot!
Best,
Sara

commented

Hey,

sorry for the delay, I needed to take care of some deadlines first :)

please make sure, that you have used some heuristic to cluster the original metastasis into individual objects otherwise you will only generate a single bounding box around all of them (since the original dataset seems to be in a semantic segmentation format).

test_labels simply indicates if you are planning to split a test set which will have test labels as well. If you there is one, set it to True otherwise to False.

target_class is targeted towards patient wise evaluation with multiple classes, you can either leave it empty or set it to 0.

Best,
Michael

commented

Dear Michael,
thanks a lot for the explanation.
Just to make sure for "test_labels": with "test set which will have test labels as well", do you mean the existence of the folder "labelsTs/" with the same classes as the training? Or something else?
So far I was running my code with "test_labels=False" and it seemed to work reasonably, now I am wondering if I missed something.
Best,
Sara

commented

Yes, if you are planning to use imagesTs & labelsTs than you should set test_labels=True, that will simply trigger the processing for the test images & labels as well (since for the evaluation of the test set the boxes need to be extracted from the test labels). nnDetection was designed for use cases where the test set labels might be unknown (e.g. during a challenge) and thus it won't automatically process the labels.

Training etc. don't use the test set - the test set is really intended as a true test set i.e. you should only predict it once at the very end of the model development. This is done by separate commends i.e. running nndet_predict and nndet_eval with the --test option.

Best,
Michael

commented

  • Practically, what would test_labels=True do? Would it create also a labelsTs/ folder under {my task}/preprocessed/D3V001_3d/ ? At the moment I only have there: imagesTr/, imagesTs/, and labelsTr/ .
    However, I already have also a labelsTs/ folder under {my task}/preprocessed/ .

  • I checked the code and it seems to me that "test_labels=True" would only additionally trigger "check_data_and_label_splitted" on the test folder, just as a check.

  • Should what I have not be enough, what do I need to rerun? nndet_prep?

Thanks and best.
Sara

commented

  • Yes, it will create labelsTs in the preprocessed folder (which is basically a conversion of the format)

  • Simply rerun nndet_prep which will prepare the test labels as well (

    nnDetection/nndet/planning/experiment/utils.py

    Line 58 in 8032b8d

    def create_labels(
    automatically checks if labelsTs exist irrespective of the dataset.json)

Best,
Michael

commented

This issue is stale because it has been open for 30 days with no activity.

commented

This issue was closed because it has been inactive for 14 days since being marked as stale.