daniel-merrick / Learning-from-Simulated-and-Unsupervised-Images-through-Adversarial-Training-SimGAN-PyTorch

PyTorch implementation of 'Learning from Simulated and Unsupervised Images through Adversarial Training'

Learning from Simulated and Unsupervised Images through Adversarial Training (SimGAN, PyTorch)

This repository is finished but documentation is being updated. Documentation and usage may be unclear right now, any questions or concerns are welcome!.

This repository is an implementation of this paper in PyTorch. Many of the repositories I found that were also written in PyTorch either were (1) buggy or (2) incomplete. This repository modifies other repos and makes this implementation more complete and hopefully easier to use.

Motivation

This paper presents a method to add realism to unreal or synthetic data, allowing us to train on a labeled set of synthetic data that matches the distribution of realistic data 'better,' resulting in better testing accuracy. Finish later.

Build status

Code base is finished! Documentation and usage information is being uploaded. More detailed results will be added eventually.

Results

Columns 1, 3, 5 and 7 are the input syntheric images (Unity Eyes).
Columns 2, 4, 6 and 8 are the refined images (Unity Eyes + Realism)

Learning Rate: 0.001
K_R: 2
Delta (penalty on reconstuction loss): 0.75
Batch Size: 512
Buffer Size: 128000
Num Steps: 100000

Installation

See docker file for the required packages & libraries.

Usage

See gaze estimator README for instructions on training and testing gaze estimator.

See simgan README for instructions on training and testing the simgan.

Credits

This project is based on this paper. The repo is largely built from automan000's repository but I fixed some of the bugs, reorganized the code, and added code to train & test a gaze estimator.