cesarsouza / synthetic-computer-vision

A list of synthetic dataset and tools for computer vision

Synthetic for Computer Vision

A list of synthetic dataset for computer vision. This is a repo for tracking the progress of using synthetic images for computer vision research.

If you found any important work is missing or information is not up-to-date, please edit this file directly and make a pull request.

See also: http://riemenschneider.hayko.at/vision/dataset/index.php?filter=+synthetic

Click publication to jump to the paper title, detailed information such as code and project page will be provided together with pdf file.

## Outline This is a brief summary of this page, you can quickly jump to what you want. - [Synthetic image dataset](#dataset) - [3D models](#models) - [Tools](#tool) - [Resource Index](#resource) - [Reference](#reference)

## 1. Image dataset [↑](#outline)

Name	Publication
Virtual KITTI	CVPR2016
Synthetia	CVPR2016
Sintel	ECCV2012
SceneFlow	CVPR2016

## 2. 3D Model Repository [↑](#outline)

Realistic 3D models are critical for creating realistic and diverse virtual worlds. Here are research efforts for creating 3D model repositories.

Name	Publication
ShapeNet	ArXiv
3dscan	ArXiv
seeing3Dchairs	CVPR2014

## 3. Tools [↑](#outline)

Name	Platform	Publication
Render for CNN	Blender	ICCV2015
UETorch	Unreal Engine 4(UE4)	ICML2016
UnrealCV	UE4	ArXiv
VizDoom	Doom	ArXiv

## Resources [↑](#outline)

ECCV 2016 Virtual/Augmented Reality for Visual Artificial Intelligence (VARVAI) workshop link

Virtual Reality Meets Physical Reality: Modelling and Simulating Virtual Humans and Environments Siggraph Asia 2016 workshop link

Misc. ↑

• RealismCNN github
• Abnormality Detection in Images(http://paul.rutgers.edu/~babaks/abnormality_detection.html)

Related topics ↑

Domain Transfer

Reinforcement Learning

People

Great research always comes from great researchers. This is a short list of researchers that are combining computer vision with virtual worlds.

Reference ↑

If you want to edit this README file. The div id is bib citekey from google scholar, use div id makes it easier to reference a work in this document.

Non publication

universe.openai.com

2016

(Total=10)

• SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth

• TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games (pdf) (code)

• A Virtual Reality Platform for Dynamic Human-Scene Interaction. 2016
  (pdf) (project)

• ResearchDoom and CocoDoom: Learning Computer Vision with Games. 2016
  (pdf) (project)

- The SYNTHIA dataset: A large collection of synthetic images for semantic segmentation of urban scenes. 2016 ([pdf](http://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Ros_The_SYNTHIA_Dataset_CVPR_2016_paper.html)) ([project](http://synthia-dataset.net/)) ([citation:4](http://scholar.google.com/scholar?cites=9178628328030932213&as_sdt=2005&sciodt=0,5&hl=en))

- Virtual Worlds as Proxy for Multi-Object Tracking Analysis. 2016 ([pdf](http://arxiv.org/abs/1605.06457)) ([project](http://www.xrce.xerox.com/Research-Development/Computer-Vision/Proxy-Virtual-Worlds)) ([citation:5](http://scholar.google.com/scholar?cites=11727455440906017188&as_sdt=2005&sciodt=0,5&hl=en))

• Playing for data: Ground truth from computer games. 2016
  (pdf) (citation:1)

• Play and Learn: Using Video Games to Train Computer Vision Models. 2016
  (pdf) (citation:1)

- ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning. 2016 ([:octocat:code](https://github.com/Marqt/ViZDoom)) ([pdf](http://arxiv.org/abs/1605.02097)) ([project](http://vizdoom.cs.put.edu.pl/)) ([citation:4](http://scholar.google.com/scholar?cites=4101579648300742816&as_sdt=2005&sciodt=0,5&hl=en))

- A large dataset of object scans. 2016 ([pdf](http://arxiv.org/abs/1602.02481)) ([project](http://redwood-data.org/3dscan/)) ([citation:6](http://scholar.google.com/scholar?cites=5989950372336055491&as_sdt=2005&sciodt=0,5&hl=en))

- UnrealCV: Connecting Computer Vision to Unreal Engine 2016 ([:octocat:code](https://github.com/unrealcv/unrealcv)) ([project](http://unrealcv.github.io)) ([pdf](http://arxiv.org/abs/1609.01326))

- Learning Physical Intuition of Block Towers by Example 2016 ([:octocat:code](https://github.com/facebook/UETorch)) ([pdf](http://arxiv.org/abs/1603.01312)) ([citation:12](http://scholar.google.com/scholar?cites=12846348306706460250&as_sdt=2005&sciodt=0,5&hl=en))

• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning 2016
  (pdf)

2015

(Total=3)

• A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. 2015
  (pdf) (citation:9)

- Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views. 2015 ([:octocat:code](https://github.com/ShapeNet/RenderForCNN)) ([pdf](http://www.cv-foundation.org/openaccess/content_iccv_2015/html/Su_Render_for_CNN_ICCV_2015_paper.html)) ([citation:33](http://scholar.google.com/scholar?cites=1209553997502402606&as_sdt=2005&sciodt=0,5&hl=en))

- Shapenet: An information-rich 3d model repository. 2015 ([pdf](http://arxiv.org/abs/1512.03012)) ([project](http://shapenet.cs.stanford.edu/)) ([citation:27](http://scholar.google.com/scholar?cites=1341601736562194564&as_sdt=2005&sciodt=0,5&hl=en))

2014

(Total=2)

• Virtual and real world adaptation for pedestrian detection. 2014
  (pdf) (citation:46)

- Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. 2014 ([:octocat:code](https://github.com/dimatura/seeing3d)) ([pdf](http://www.cv-foundation.org/openaccess/content_cvpr_2014/html/Aubry_Seeing_3D_Chairs_2014_CVPR_paper.html)) ([project](http://www.di.ens.fr/willow/research/seeing3Dchairs/)) ([citation:110](http://scholar.google.com/scholar?cites=18030645502969108287&as_sdt=2005&sciodt=0,5&hl=en))

2013

(Total=1)

• Detailed 3d representations for object recognition and modeling. 2013
  (pdf) (citation:67)

2012

(Total=1)

- A naturalistic open source movie for optical flow evaluation. 2012 ([pdf](http://link.springer.com/chapter/10.1007/978-3-642-33783-3_44)) ([project](http://sintel.is.tue.mpg.de/)) ([citation:227](http://scholar.google.com/scholar?cites=15124407213489971559&as_sdt=20000005&sciodt=0,21&hl=en))

2010

(Total=1)

• Learning appearance in virtual scenarios for pedestrian detection. 2010
  (pdf) (citation:79)

2007

(Total=1)

• Ovvv: Using virtual worlds to design and evaluate surveillance systems. 2007
  (pdf) (citation:58)