KimRass / ILVR

PyTorch implementation of 'ILVR' (Choi et al., 2021) from scratch and applying it to 'DDPM' on CelebA at 64 × 64

1. Pretrained Models

• Download 'ddpm_celeba_64×64.pth' from DDPM

2. Samples

1) Sampling from Single Reference

mode="single_ref", dataset="celeba", scale_factor=4
The top left is each reference image.

mode="single_ref", dataset="celeba", scale_factor=8
The top left is each reference image.

mode="single_ref", dataset="celeba", scale_factor=16
The top left is each reference image.

mode="single_ref", dataset="celeba", scale_factor=32
The top left is each reference image.

mode="single_ref", dataset="celeba", scale_factor=64
The top left is each reference image.

2) Sampling from Various Scale Factors

mode="various_scale_factors", dataset="celeba"
The leftmost is each reference image and the rest correspond to scale_factor=4, 8, 16, 32, 64 from left to right.

3) Sampling from Various Conditioning Range

mode="various_cond_range", dataset="celeba"
The leftmost is each reference image and the rest correspond to ILVR on steps from 1000 to 0, to 125, 250, 375, 500, 625, 750, 875, 1000 (No ILVR steps) from left to right.

2. Implementation Details

3. Experiments

1) Image Resizing Modes

Original image

Nearest	Area	Bilinear	Bicubic

• Upsample과 Downsample 시에 동일한 Mode를 사용하면 Artifact가 발생합니다.

Resizeright [1]	Area-Bicubic
이미지에 자연스럽게 Blur를 적용한 듯한 효과가 생깁니다.	Downsample시에 Area mode를, Upsample시에 Bicubic mode를 사용하면 이와 거의 유사한 효과를 낼 수 있습니다. 이쪽이 더 간단하게 구현되므로 본 구현체에서는 이 방법을 사용했습니다.

4 Theoretical Background

$${x^{\prime}{t - 1}} \sim p{\theta}(x^{\prime}{t - 1} \vert x{t})$$ $$y_{t - 1} \sim q(y_{t - 1} \vert y)$$ $$x_{t - 1} \leftarrow \phi_{N}(y_{t - 1}) + x^{\prime}{t - 1} - \phi{N}(x^{\prime}_{t - 1})$$

5. References

• [1] https://github.com/assafshocher/ResizeRight