Why using only the first generated mask ?
ClementAlba opened this issue · comments
Hello,
I saw in your file SAMTools.py that you are using only the first mask produced by SAM :
Line 130 in fb237ad
By default, SAM generate 3 masks per prompt with a score for each one. Maybe choose the mask with the highest score can be a better approach ? Even better, let the user choose which mask he wants !
Hi ClementAlba, thanks for your interest. We are using only the first mask because we have set the multimask_out option as False (see a few lines above), so the predictor will only output one mask each time.
The multimask_out option is designed to handle ambiguity problems when the user only gives one single-point prompt. But our tool aims to help users interactively refine the mask results by giving additional prompts needed in real-time. So the ambiguity problem should be much rarer in practice. A quote from the SAM's original code comments "For non-ambiguous prompts, such as multiple input prompts, multimask_output=False can give better results."
You can get more information on page no.17 of the SAM paper.