Virtual Try-On testing on demo
carvychen opened this issue · comments
Hi, thank you for your great work.
How to test the cases of virtual try-on with your demo where only one control input, but at least there are two conditions, e.g., cloth mask and garment image?
We have yet to release that part. A rough pipeline: a) uses segmentation to get cloth masks, b) gets the canny edge of the model, c) seecoder + ControlNet, gets a new image, d) combines new images and original image based on the mask.
Hello, thank you for providing the code, but I am currently prompted with torch. cuda. OutOfMemoryError: CUDA out of memory when starting. The details are as follows: torch. cuda. OutOfMemoryError: CUDA out of memory Tried to allocate 4.50 GiB (GPU 0; 4.00 GiB total capacity; 3.16 GiB already allocated; 0 bytes free; 3.40 GiB reserved in total by PyTorch) If reserved memory is>>allocated memory try setting max_ Split_ Size_ Mb to avoid fragmentation See documentation for Memory Management and PYTORCH_ CUDA_ ALLOC_ CONF. How to solve it?
We have yet to release that part. A rough pipeline: a) uses segmentation to get cloth masks, b) gets the canny edge of the model, c) seecoder + ControlNet, gets a new image, d) combines new images and original image based on the mask.
Hello, thank you for your great work.
I'm not sure if the demo we use following your pipeline b) and c).The result looks not so good. Should we use the realistic sd model like koreandoll? In addition, the canny edge of the ControlNet preprocess seems to have some influence，should we get canny by hand？
It looks that we should modify the ControlNet to inpainting, like https://github.com/mikonvergence/ControlNetInpaint
Do not know how difficult is it to adapt the code.
We have yet to release that part. A rough pipeline: a) uses segmentation to get cloth masks, b) gets the canny edge of the model, c) seecoder + ControlNet, gets a new image, d) combines new images and original image based on the mask.
Hello, thank you for your great work. I'm not sure if the demo we use following your pipeline b) and c).The result looks not so good. Should we use the realistic sd model like koreandoll? In addition, the canny edge of the ControlNet preprocess seems to have some influence，should we get canny by hand？
Please check the canny edge; Looks like these texture-edges dominate, which is the expected behavior. Also you need d) copy the face and other parts of the image into the generated image. (Inpainting maybe a more desirable approach)
Hi all, I'm the author who makes the virtual try-on part.
Here shows how we generate the cases presented in our paper. A very simple pipeline:
a) Generate the cloth mask using SAM via a point prompt. We used the WebUI at https://huggingface.co/spaces/AIBoy1993/segment_anything_webui
b) Remove white borders around the mask and regenerate the mask area via SeeCoder together with either canny (preprocess required) or seg (preprocess not required) ControlNet. (P.S. You may need to vary CFG and seeds for better visual effects)
c) Replace the original cloth according to the mask. Then you can get similar results to our paper figure.