Giters

VinAIResearch / CPM

💄 Lipstick ain't enough: Beyond Color-Matching for In-the-Wild Makeup Transfer (CVPR 2021)

Home Page:https://thaoshibe.github.io/CPM

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Unsatisfied results tested on customized images

BIGJUN777 opened this issue · comments

commented

Hello

I got some non-ideal results when testing the model on other images.

  1. As shown in the following image, there are some distinct boundaries on the resulting image. Any idea on this?

    image

    When diving into the code, I found you applied Face Detection before PRNet (See here). Because the image I input has been cropped, I commented those codes related to Face Detection in process() function (See below). However, the result turned out to be worse. It seems this Face Detection is necessary. Would you provide me with more explanation on this?

    image

    image

  2. When testing on my device (Tesla V100), it took several seconds to process an image, which is really time-consuming. Is this normal? If so, why will it be such time-consuming (it seems the model is not really complex)?

Looking forward to your reply.
Thanks.

commented

Thanks for your interest in our work. I'm happy to help!

  1. About the boundaries
    Currently, we employed the pre-trained model of PRNet. As PRNet is not a 3D full-face model, it is expected that there will be artifacts in boundaries (Fig. 1). For better results, I suggest:
    1.1. Using image blending technique
    1.2. Employ other 3D full-face models. You can easily use any other 3D full-face models to replace PRNet in our framework.


Fig. 1. Reconstructed face from PRNet

  1. About face detection
    Using face detection and cropped face ensure the quality of the PRNet (See more Issues about that here). So yes, you can comment out those codes, but it will affect the performance of the 3D face model, subsequently cause unpleasant results (as you experienced).

  2. Testing speed
    I tested on a single RTX 1080Ti, running time ~ 1-2 seconds per image. The slowest part of the code is the 3D parts L34-36. I'd recommend you to check whether the code is currently run on GPU or not (As maybe tensorflow-gpu is not working).
    After that, further details to speed up the 3D model parts can be found on PRNet's repo.

Feel free to ask if you have other questions ( ❛ᴗ❛ )

commented

Thank you for your detailed as well as useful reply.

For the testing speed, as you mentioned, there is something wrong with the TensorFlow-GPU in my env. Will fix it later.