Shadow suppression
DonaldKam opened this issue · comments
It a marvelous job and I noticed that the motion of shadow was suppressed in the supplementary video shown as following
This differentiates from conventional methods like Lucas-Kanade. Can you explain why？
There is no explicit shadow treatment in FlowNet2 — every behavior is data-driven. The network has enough capacity to learn something about shadows from the available KITTI groundtruth during finetuning (after pretraining on a larger dataset to learn general flow features).
FlowNet2 has a relatively large receptive field, so it can "analyze" the area around the car and shadow and make decisions based on that.
Classical methods, especially purely local ones like Lucas-Kanade, do not learn correlations between appearance and flow. Consequently, they can never distinguish a black object from a black shadow. A neural network can learn that if black blobs and objects often appear together, then a black blob could be a shadow.