Request
mashreve opened this issue · comments
Thank you very much for the code!
A quick question: is it possible to get a full resolution frontalized face, or is it constrained to the size of the CNN input (e.g., 160)? In other words, if I have a source image that contains a 500x500 pixel face, is it always going to be downsized to 160 x 160 after frontalization?
Thank you,
MS
Hi @mashreve ,
thank you for your interest in the project.
First of all, you can remove the resize option using the configuration file.
Second, the rendered is not constrained to the size of the CNN input but is constrained by the unprojected matrix we are using.
In particular:
- the renderer resolution is bounded to the size of the unprojected matrix that we load offline from the
.matfiles. This matrix is created a priori so it can't be modified online: this is the reason why our renderer is also very fast and does not need OpenGL. - additionally, even though you may render at higher resolutions by changing the unprojected matrix, probably the CNN software, you may be using, is going to resize the image to the template size of the CNN input, unless you are doing random cropping on the input image, which we did not in our implementation.
For more information on how to generate unprojected matrix see this project page
Hope this helps,
Best,
iacopo
Your project is so useful!
is it possible to get result of more poses? Right now it can only generate pose of 0 40 75 around y-axis. Could you kindly provide model of rotation around x-axis?
Thank you very much.
Hi @someone9388,
thanks for your interest in the project.
Yes, it is possible to add more poses even in pitch; currently we are working on it.
If you want, you can try to generate your own model starting from this project page
Hope this helps,
Best,
iacopo
Awesome project!
I have the same confusion as @someone9388, wondering how to generate model of more poses. I tried the code in here(the page you provided), but it only generate unprojected model of a fix pose while projected images' poses vary.
Also it seems hard to generate models in the same format as the .mat files you provided. But this is what I need. Is there any way to reach this?
Many thanks.
Hi @thu-zxs,
Thanks, we are glad this project is helping the community!
The code I pointed you is a starting point but I understand that a bit more of effort is needed to save the data in the same format we are using.
Moreover, the code does generate multiple poses: you just have to vary the parameter in the rendering function to render a different yaw, or pitch etc.
Finally, regarding the release of the code that generates the mat files, we did not plan to release it soon, but we may do it later on.
Hope this helps,
Cheers,
iacopo
@iacopomasi @TalHassner Thanks!
”Second, the rendered is not constrained to the size of the CNN input but is constrained by the unprojected matrix we are using.“
I want to know what is unprojected matrix???
Is it the camera matrix？
No, it's not.
unproject is a name we've been using to call a matrix of the same dimensions as the output image, but instead of RGB values, for each pixel it has the 3D coordinates of the surface point on the 3D face model, projected onto that pixel (hence "unproject"). This data structure is sometimes referred to by others as a "geometry map". A separate unproject matrix is required for every output view you want to render to (e.g., profile, frontal, etc.) and for every 3D face model you want to use. So, if you always render to frontal with the same 3D face model, you only need one such matrix.
The unproject matrix is the result of projecting 3D onto the 2D viewpoint. This step is a standard part of any rendering pipeline (e.g., if you ever use OpenGL). We precompute this matrix, thus saving a lot of runtime repeatedly computing it every time we render a face to our fixed output views (frontal, half-frontal, profile etc.).
The matrix itself can be produced using a variety of rendering libraries, but we used the OpenSceneGraph renderer packaged for MATLAB. Take a look at that webpage for a code snippet which I think will help explain things.
Yours,
Tal