You can convert text to face image!!! Thanks to @Puzer and @pbaylies for the original style Encoder
Short explanation of encoding approach:
- Bert is used for transforming sentence to embedding
- Original pre-trained StyleGAN generator is used for generating images
- Pre-trained ResNet network is used for transforming a reference image and generated image into high-level features space
- Simple Model is used for transforming Text Embedding and image into high-level features space
More examples you can find in the Jupyter notebook
You can generate latent representations of your own images using two scripts:
- Extract and align faces from images
python align_images.py raw_images/ aligned_images/
-
Download the stylegan pre-train model (百度云链接), put the stylegan.pkl to model folder
-
Training ResNet50 encoder train your own with trainResnet.py or download my pre-trained model! Put the model in model/finetuned_resnet.h5
Origin github -
Find latent representation of aligned images
python encode_images.py aligned_images/ generated_images/ latent_representations/
You can generate sentence embedding with bert-as-service
- Install bert-serving
pip install bert-serving-server
pip install bert-serving-server
- Start the BERT service
bert-serving-start -model_dir ./model/chinese_L-12_H-768_A-12 -num_worker=4
bert chinese model download address 百度云链接
You can describe face image by yourself, like this:
48889.png 黑色短发年轻女子，有刘海，开心的笑，露齿，大眼睛
48872.png 黑色短发青年男子，戴墨镜，侧面
Used image from FHHQ dataset
The model is simple with some Dense layer
python trainTextToFace.py