Training with custom data gives unexpected results.
iamrishab opened this issue · comments
I tried to fine-tune the given checkpoints with custom data. I had prepared the required data pipeline after taking reference from the given training data. Although I kept everything in the train.py almost same, I am getting a blank image as output during inference.
I have cross-checked the data preparation and found that the differenceMask I am calculating, comes out be an image of 0s for all cases.
Kindly let me know if there is something I might be missing in the steps. TIA!
Fine-tuning Gan's is something I don't recommend, you need to train from scratch. You can get the data from cp-vton link, and try to replicate the results.
Gan learns a particular distribution since GAN's are trained in an adversarial fashion which helps them to replicate certain underlying patterns in a dataset.
Also, we present a concept rather than a system, so you need to work on your end. We were able to use this concept in 4 different tasks infact.
@nile649 Great work! Thanks for sharing.
I understood your point and trying to outwork something around it. Actually, I was trying to use it for cloth warping which I believe is the primary focus of this paper. Kindly let me know if you know some good opensource project on this which is generalizable enough to get satisfactory results on a custom dataset of different cloth types of both men and women.
In my research, we didn’t find anything which can give a perfect result. My work gives good results, but again it’s not the best, and even current papers from CVPR are pretty much the same in metric comparisons.
It pretty much depends on your application, I can help you to train your own model, but to be honest, it's a pain in the ass doing a custom model.
Body parsing, Pose estimation, Segmentation network, and then Poly-GAN model or any other model.
I have tried with different warping models from latest SOTA papers as well. But the applications of these papers seems limited to only few cloth types like shirts or t-shirts. Given the diversity of cloths of men and women, I believe no single model is able to do justice in warping. Tried training with a subset of my dataset as well, but still it is not able to learn other different cloth types. Looks like we need more diverse data samples to benchmark the models and a more novel architecture.
It won't, the diversity of clothes also means the diversity of distribution of data. GAN's aren’t meant for such complex tasks.
At least not for now. Also, the output from a GAN is not that crisp as we are sampling from a large distribution, which affects the real experience of a VTO. Anyways, I am working on an alternative approach. Thank you for your inputs. Appreciate it.