Obtaining segmentation masks for localized representation learning

[sonia rao]

Incorporating segmentation masks into appearance module will allow learning of localized ciliary patches. Right now, we have segmentation masks on ~20% of entire dataset and appearance module is not targeting cilia regions. With respect to time constraints, unknown segmentation masks can be computed through 1) Rudimentary thresholding using pixel values, optical flow, and/or derivative quantities or 2) learned via supervised NN/ML algorithm trained on existing segmentation masks, optical flow quantities, and/or derivative quantities.

Ideally, we would be able to obtain segmentation masks without supervision through a "refinement" stage that occurs in the larger appearance pipeline. However, I'm simply a novice and do not know how to do that yet; having a full set of segmentation masks as an initial sanity check for the appearance pipeline has worthwhile short term benefits.

Next steps:

• Create set of segmentation masks via thresholding, optimizing for minimal false positives
• If ^ are inadequate, train NN to learn segmentation masks

Eventually:

• modify appearance module to iteratively refine rough thresholded/unsupervised segmentation masks as a byproduct of spatial reconstruction

[Meekail Zain]

This issue should be transformed into a milestone with individual issues created to reflect the different steps as they appear