This repository, generates photorealistic face synthetic data, as well as the the corresponding depthmaps and 3D face structure/mesh using plug & play notebooks inspired by EG3D. In essence, EG3D is leveraged to generate synthetic face data and extract the respective face meshes, generated by EG3D or extracted from the corresponding depthmaps. The generated EG3D .obj meshes have higher facial detail, in contrast to the meshes extracted from the underlying 128x128 depthmaps. Below you can find the relavant notebooks:
| Description | Link |
|---|---|
| Generate EG3D face data (image, depthmap and 3D .obj) |  |
| Visualize synthetic face mesh | |
The notebooks above, include detailed steps and instruction on generating synthetic EG3D data of fake photorealistic faces. Additionally, we project real images, to EG3D's latent space (PTI-inversion), to reproduce-generate custom real images, as high-quality EG3D faces (with the corresponding depthmap and 3D face structure). Finally the output mesh files (.obj, .ply, .mrc) of the corresponding generated face images, can be further visualized.
This repository is inspired by the EG3D paper and uses the codebase from EG3D. Please refer to the project's webpage for more information.
By leveraging EG3D, the synthetic training set for training our depth & latent code regressor models was achieved. The training set consists of 50,000 samples of high-quality face images (.png) (512x512), the corresponding depthmaps(.png) (128x128), the 3D meshes (.obj), the EG3D latent code (.npy) and the corresponding camera parameters (.npy) (25,1) (16 extrinsic and 9 instrinsic parameters). The training dataset, to be generated requires 21 GPU days on a Tesla P100 GPU. The training dataset link with the 3D face shapes (.ply), the depth image, EG3D latent code and synthetic high quality face images can be downloaded using the link here. The training dataset generated face images, from various viewpoints, to ensure the robustness of the trained models. The various viewpoint categories and the respective dataset percentages are depicted in the image above.
The inversion method for EG3D is inspired and heavily borrows from the PTI-inversion method implementation from the following repository, found here. Our own implementation of EG3D inversion, leveraged in the notebooks, can be found here. An evaluation dataset called "Prosopo" is also created using this PTI inversion method and consists of real face images, along with their corresponding synthetically generated equivalents.