Xiang-Pan / NYU_CV_Project

Sim4Real: Simulation Data for Realistic Semantic Segmentation Model

Xiang Pan (xp2030), Yuxuan Sun (ys4375)

Check our slides

Models and Datasets

All the simulation trained models are stored in cached_models and cached_datasets. The model loader and dataset definition can be found as task_models and task_datasets.

We will sort our datasets and release via goold drive.

How to run the code

you can check the ./scripts for locally running. And you can also use scripts in ./slurm_scripts to run in HPC of NYU.

Example

Training from scratch

python main.py --gpu 0 --task_name=KITTI

Transfer from simulation learned model

python main.py --gpu 0 --task_name=KITTI --load_checkpoint_path=cached_models/unity-streetview-low-res_mIOU=0.77.ckpt --log_name=streetview-low_kitti

Dataset Generation

Our generation pipeline is based on https://github.com/Unity-Technologies/com.unity.perception and https://github.com/Unity-Technologies/Unity-Simulation-Smart-Camera-Outdoor.