Training the model, train vs. test data

[Corey Crossen]

Hello!

I have been looking at the config_sintel.py file a bit and am not understanding how the train/test data are separated. In your slides, you suggest that this model can be trained using exclusively the retouched image data, but the train.py function seems to expect matched pairs of images with the untouched data being in one folder, the retouched data in a second folder... What about the test data? In the config_sintel.py file, the fake_data_provider_test folder is the same as the training folder, while the comment above it specifically says not to do this.

Right now I am thinking I should have three folders in my model directory, like this:

cfg.fake_data_provider = lambda: FolderDataProvider(folder='models/ColorCorrect/Uncorrected', default_batch_size=cfg.batch_size)
cfg.fake_data_provider_test = lambda: FolderDataProvider(folder='models/ColorCorrect/Validation', default_batch_size=cfg.batch_size)
cfg.real_data_provider = lambda: FolderDataProvider(folder='models/ColorCorrect/Corrected', default_batch_size=cfg.batch_size)

Where the Uncorrected Folder has 300 untouched images, the Validation Folder has 50 different untouched images, and the Corrected Folder has a total of 350 images, all matching to either the contents of the Uncorrected or Test Folders.

Is this the correct approach?

Thanks for your time, and for sharing your very impressive work!

[Corey Crossen]

From the paper:

Datasets.We utilize two sources of training data:
•The MIT-Adobe FiveK Dataset. Bychkovsky et al. [2011]compiled a photo dataset consisting of 5,000RAW images and retouched versions of each by five experts. In this work, we randomly separate the dataset into three parts: (part 1) 2,000input RAW images, (part 2) 2,000 retouched images by retoucher C, and (part 3) 1,000input RAW images for testing. The three parts have no intersection with each other.