Help with some implementation details
lvaleriu opened this issue · comments
Hello! I'd like to ask you some hint about implementing the following: i'd like to train a segmentation model using 2 inputs (an image input & a mask input) instead of one (image input). So there are 2 "different" things from the classic fastai v2 examples: 2 inputs instead of one & the unet learner receives 2 inputs - can i specify at what point i can insert the mask? (i'd like to insert it towards the end of the model, just before the classification of each pixel).
I manage to do that easily the keras functional api (i just declare 2 inputs, implement a generator and an unet and then concatenate the second input and some layer in the unet just before the last classification layer), but i'd love to stay with fastai v2.
Thank you!
(I've seen that you are doing many un-orthodox things so I took a bet that you might be able to help me with this)
I’m unsure to how you’d go about that. You may want to post on the vision forum in fastai2 for ideas. You could declare three blocks and have 2 be your inputs and then work with it from there.
IE: blocks=(ImageBlock, MaskBlock, OutputBlock) and tag it with a number of inputs. I believe this would be similar to the bounding box example.