visinf / irr

hi, I was trying to train my own datasets and use the MultiScaleEPE_PWC loss, but something is wrong as follows:

during debugging, I found in class MultiScaleEPE_PWC:

used the

but 'output' is a list containing 4 tensors, not a single tensor which has no attribute 'size()' , this is different from other loss class.
Could you tell me whether I should change '_downsample2d_as(target, output_ii)' into '_downsample2d_as(target, output_ii[ii])'?
I see in other loss class, '_downsample2d_as(target, output_ii[jj*2])' is used, what's the meaning of '*2'?

Please refer to the model description file. Each list contains forward & backward optical flow, before and after the refinement layer. It's also different at each pyramid level.

irr/models/IRR_PWC.py

Line 147 in dacd07b

flows.append([flow_cont_f, flow_cont_b, flow_f, flow_b])

irr/models/IRR_PWC.py

Line 153 in dacd07b

flows.append([flow_f, flow_b])

Thanks a lot for your quick reply. Sorry I haven't make it clear. Your have written a quite clear logic in 'IRR_PWC.py' and I think I have known that there are 7 layers, within which 5 layers contain [flow_cont_f, flow_cont_b, flow_f, flow_b], the last 2 layers contain
[flow_f, flow_b]. However, in MultiScaleEPE_PWC loss, the code is not right, output_ii is not a tensor(it's a list that consists of 4 or 2 tensors), so couldn't be computed with 'target'(it's a single tensor) by using _downsample2d_as function.
Through debugging, I know that '2*' means you take out the forward flow, but I am not clear how to use context flow and flow, sometimes use context flow and sometimes use four of it? Maybe I need to read the paper carefully

Please use this loss function, MultiScaleEPE_PWC_Bi_Occ_upsample, for training the IRR-PWC model. It requires ground truth for bi-directional optical flow and occlusion maps for each view.
The loss function you used, MultiScaleEPE_PWC, is for training vanilla PWC-Net (https://github.com/visinf/irr/blob/master/models/pwcnet.py).

hmm, I have tried MultiScaleEPE_PWC_Bi_Occ_upsample for IRR-PWC, yes it worked well. Ok, I will try PWC-Net. Thanks a lot.
By the way, I have a question to ask for your help: the ground-truth flow should be divided by 20 first, just as the div_flow in IRR-PWC.py ?

Yes empirically, dividing the GT by 20 helps for achieving better accuracy. (it's just following the practice from FlowNet 1.0)
But, It's actually up to you, and I think there can be a better setup.

Ok， thanks a lot.

MultiScaleEPE_PWC loss not right