gladcolor / SfMNeXt-Impl

SfMNeXt: The NeXt Series of Learning Structure Prior from Motion

👩‍⚖️ Demo

Online demo is available at HERE

👀 Training

To train on KITTI, run:

python train.py ./args_files/args_res50_kitti_192x640_train.txt

For instructions on downloading the KITTI dataset, see Monodepth2

To train on CityScapes, run:

python train.py ./args_files/args_cityscapes_train.txt

To finetune on CityScapes, run:

python train.py ./args_files/args_cityscapes_finetune.txt

For preparing cityscapes dataset, please refer to SfMLearner's prepare_train_data.py script. We used the following command:

python prepare_train_data.py \
    --img_height 512 \
    --img_width 1024 \
    --dataset_dir <path_to_downloaded_cityscapes_data> \
    --dataset_name cityscapes \
    --dump_root <your_preprocessed_cityscapes_path> \
    --seq_length 3 \
    --num_threads 8

💾 Pretrained weights and evaluation

You can download weights for some pretrained models here:

Methods	WxH	abs rel	RMSE
KITTI (ResNet-50)	640x192	0.088	4.175
KITTI (ResNet-50)	1024x320	0.082	3.914
CityScapes (ResNet-50)	512x192	0.106	6.237
KITTI (ConvNeXt-L)	1024x320	0.043	1.698

To evaluate a model on KITTI, run:

python evaluate_depth_config.py args_files/hisfog/kitti/resnet_320x1024.txt

Make sure you have first run export_gt_depth.py to extract ground truth files.

And to evaluate a model on Cityscapes, run:

python ./tools/evaluate_depth_cityscapes_config.py args_files/args_res50_cityscapes_finetune_192x640_eval.txt

The ground truth depth files can be found at HERE, Download this and unzip into splits/cityscapes.

🖼 Inference with your own iamges

python test_simple_SQL_config.py ./args_files/args_test_simple_kitti_320x1024.txt

Future Works

• release code for training in outdoor scenes (KITTI, Cityscapes)
• model release (KITTI, Cityscapes)
• code for training in indoor scenes (NYU-Depth-v2, MannequinChallenge)
• code for finetuning self-supervised model using metric depth
• model release for indoor scenes and metric fine-tuned model

License

All rights reserved. Please see the license file for terms.