function: inpaint_background_and_modify_merged_mask
ByChelsea opened this issue · comments
Hi, Lee. Thanks for your great work!
When I read the code, I didn't quite understand the function inpaint_background_and_modify_merged_mask. Is this intend to prevent the color of the focus area from spilling over into the foreground and background? Could you please briefly explain the purpose of this function? Thank you!
By the way, I guess there's a condition missing here. Because it might also need inpaint_flag to be True.
Thanks again and looking forward to your reply!
Hi, @ByChelsea.
It took me some time to review the code as it has been a while since I developed this code.
- The
inpaint_background_and_modify_merged_mask()function is to handle occluded regions, especially when foreground plane has larger CoC than the background CoC. In the function, we first compute a mask to inpaint, then inpaint the merged image leveraging the mask. You may want to visualize each step for better understanding. inpaint_flagis not used in the code. The flag is from the legacy code. As inpainting is needed after we reach the focused plane (i.e. when CoC is 0), the current condition is enough.
Best,
Thank you very much for your reply after reviewing the code!
I know this is very troublesome, I hope I didn't particularly disturb you.
I don't really understand this sentence: “The function is to handle occluded regions, especially when foreground plane has larger CoC than the background CoC”. What's occluded regions and what happens if the foreground has larger CoC than the background?
No worries!
Imagine a center plane (in terms of depth of a scene) is focused.
If the focused plane is occluded by foreground planes that are placed at the front of the focused plane, we need to inpaint the occluded region of the focused plane, as the foreground planes will be blurred and the blurred region requires information from the occluded region.
"the foreground planes will be blurred and the blurred region requires information from the occluded region", from my understanding, blurred region refers to the foreground planes and occluded region refers to the focused plane. Why does the blurring of the foreground require information about the focused plane? Aren't they independent?
When the foreground plane is blurred, the portion of the occluded region (or even the whole region) becomes visible.
I attach the extreme case in the following. You can see through the focused person originally occluded by a handrail.
Ah, I see. That's a very intuitive explanation, thank you!
In fact, I thought that was the problem at first. link So does the code need to be tweaked a bit?
These lines are required, as we wand inpainted result to be normalized, otherwise, there will be unnatural seam around the inpainted regions.
I see, thanks for your reply.