This repo is all about optical flow. We are reimplementing FlowNet and FlowNet2 from scratch in Pytorch, as well as a Transformer-based version of FlowNet.
All code was run in a conda environment. To replicate the exact environment, run the following commands in the Anaconda prompt:
conda env create -f environment.yml
conda activate flowbots
For more information on installing conda environments, see https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-from-an-environment-yml-file. Alternatively, the minimum essential packages are included in a requirements.txt file. This can be installed using pip
pip install -r requirements.txt
We use the following optical flow datasets:
- Flying Chairs
- Flying Things 3D
- Sintel
The FlyingChairs dataset must be downloaded and organized in the following manner:
root
FlyingChairs
data
00001_flow.flo
00001_img1.ppm
00001_img2.ppm
...
FlyingChairs_train_val.txt
The root file must be designated in the accompanying config.json file by setting the "flying_chairs" field
For more information see https://pytorch.org/vision/stable/generated/torchvision.datasets.FlyingChairs.html
The FlyingThings3D dataset must be downloaded and organized in the following format:
root
FlyingThings3D
frames_cleanpass
TEST
TRAIN
frames_finalpass
TEST
TRAIN
optical_flow
TEST
TRAIN
The torrent files can be obtained from https://academictorrents.com/userdetails.php?id=9551. We recommend only downloading the necessary files, as the downloads are quite large. The root file The root file must be designated in the accompanying config.json file by setting the "flying_things_3d" field.
The Sintel dataset must be downloaded and organized in the following format:
root
Sintel
testing
clean
scene_1
scene_2
...
final
scene_1
scene_2
...
training
clean
scene_1
scene_2
...
final
scene_1
scene_2
...
flow
scene_1
scene_2
...
The root file must be designated in the accompanying config.json file by setting the "sintel" field
For more information see https://pytorch.org/vision/stable/generated/torchvision.datasets.Sintel.html