Learn from Correspondences instead of aflow

[Shrutheesh Raman Iyer]

Hello,

While training the reliability AP loss, is it possible to feed correspondences instead of aflow? This is because the optical flow is mostly dense while the correspondences are sparse in nature. Have you tried using correspondences to train the network? Would this affect the sampler and the AP loss?
Thanks!

[humenbergerm]

Hi! We have not tried that extensively (unless I missed it) but we would be interested in the performance. Best, Martin

[Jerome Revaud]

Hi. Same answer. Actually, R2D2 was really meant from the start to be trained from dense correspondences.
I guess it would be possible to train it from sparse correspondences, because there is a correspondence validity mask anyway that indicates which correspondence is valid or not, but we never trained it this way. I am not sure it would converge as well as before, because there would be much less samples in the end...

[Hichem Abdellali]

Hi @ShrutheeshIR

I would be curious to find out if you managed to train your network from sparse correspondences?
Thanks

[Shrutheesh Raman Iyer]

Hi @djidan10 no I did not get to it. I realized that it might be counter intuitive to train using sparse correspondences, since the problem would cease to be self-supervised in some sense, and be more supervised.

@humenbergerm and @jerome-revaud thank you for your response, apologies I must have missed it!
Thanks