[Question]During the prediction process, a command that loads only "model_best.ckpt" cannot be executed by command

[Lynclock]

❓ Question

Hello
During the prediction process, a command that loads only "model_best.ckpt" cannot be executed by command，as described below

predict_dir(source_dir=source_dir,
target_dir=prediction_dir,
cfg=cfg,
plan=plan,
source_models=training_dir,
num_models=num_models,
num_tta_transforms=num_tta_tralnsforms,
model_fn=load_all_models,
restore=True,
case_ids=case_ids,
**cfg.get("inference_kwargs", {}),
)
By default, the “load_all_models” parameter is loaded in model_fn，but this parameter loads all models and does not support model selection，lead to “num_models=1” parameter does not take effect. as described below

def load_all_models(
source_models:Path,
cfg:dict,
plan:dict,
*args,
**kwargs,
):
The above function cannot pass the argument num_models,but "load_final_model" pass the argument and identifer(str="last" or "best").

I want to ask, how can I execute the "model_best.ckpt" model with instructions without modifying the source code?

Thank you!
Lynclock

[Michael Baumgartner]

There are two different use cases here:

1. prediction from a single fold -> the easiest way is to copy the last model into a separate folder, so load_all_models will only load the best one
2. prediction from consolidated fold -> there is a ckpt option in the consolidate command which can be used to switch to the best model checkpoints.

One caveat is, that the predictions were generated with the last model by default. It is necessary to rerun nndet_sweep while sweep_ckpt="best" is used in the config to get the best inference parameters in both cases.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.