oms-Diffusion

This repository is the official implementation of OMS-Diffusion

OMS-Diffusion: One More Step Diffusion is All You Need for Virtual Try-on

News

🔥 [2024/2/28] 本项目支持IPadapter-faceID 结合 controlnet_openpose！你能通过肖像与姿势参考图进行试穿。在我们的测试中，肖像相似度 FaceIDPlus > FaceIDPlusV2 > FaceID, 所以IPadapter-faceID默认版本将会从FaceIDPlusV2转为FaceIDPlus.

We support IPadapter-faceID with controlnet_openpose！You can try virtual try-on by uploading a portrait and a reference pose image. In our testing, the portrait similarity among the three version is FaceIDPlus > FaceIDPlusV2 > FaceID, so we change the default FaceIDPlusV2 version to FaceIDPlus.

Try gradio_ipadapter_openpose.py

🔥 🔥 [2024/2/23] 现在本项目支持IPadapter-faceID！你能通过肖像进行试穿。增加图像高度能避免大头娃娃情况。

We support IPadapter-faceID now！You can try virtual try-on by uploading a portrait. Enlarge height will alleviate weird big-head result.

Have fun with gradio_ipadapter_faceid.py

Demos

IPadapter-faceID + controlnet_openpose demo

IPadapter-faceID demo

generative demo

controlnet demo (openpose, inpainting)

some unexpected funny application (need more effort)

Tips

本项目致力于协助中小服装商家解决约拍模特导致的高昂成本问题。只需一张服装图片，可以生成各种身穿该服装的模特。后期也会为个人用户提供服装试穿功能。

This project is committed to assisting small-sized clothing businesses in solving the high cost problem caused by model appointments. With just one clothing image, various models wearing that clothing can be generated In the later stage, it will also be provided for individual users for picking suitable garment.

Guidance

1、默认的提示词是"a photography of a model", 你可以通过增加词汇像 "sunglasses" 或者 "red skirt" 来设定模特，或者把"model"换成"man"来指定性别，但是作者也不清楚它对最终的试衣结果有何影响。

The default prompt is "a photography of a model". You can add words like "sunglasses", "red skirt" or change "model" to "man" to specify the model, but we do not know how it will affect the final result.

2、512权重的默认的图像分辨率是（512，384），此分辨率可能会生成扭曲的面部。你可以增加图像分辨率来获得优秀的细节，但是有些情况会导致衣服失控,需要多次尝试不同随机种子才能得到较好的结果。

The default image resolution for 512 weights is (512,384), which may generate distorted faces. You can increase the image resolution to obtain excellent details, but in some cases, it can lead to clothing losing control.

3、如果你发现衣服细节和你期望的对应不上，请尝试不同的随机种子。

If you find that the details of the clothes do not match your expectations, please try different random seeds.

4、guidance Scale来控制试衣服装的强度，但较大的值会导致图像失真。推荐2.0左右。

Guidance Scale is used to control the strength of clothes, but larger values may cause image distortion. (1.0 , 3.0) may be a good interval.

5、本项目内置了一个将服装从背景分离出来的权重，但是它可能不是最优的，如果你有更好的权重，像SAM，请替换它。

This project contains a module that separates clothing from the background, but it may not be optimal. if you have better weights like SAM. Please replace it.

Anyway, have fun with it.

WEIGHTS

512 resolution weights: huggingface or Baidu Drive

该权重是一个实验性质的版本，在VITON-HD一万多张训练集（女模特，上半身，夏装，4:3分辨率）上训练，所以它对某些颜色或类别可能有所偏好。训练分辨率为（512,384），对较小的字体可能会导致模糊变形（在高分辨率下已解决）。

This weight is an experimental model trained on over 10000 VITON-HD training images (female models, upper body, summer clothing, 4:3 resolution), so it may have preferences for certain colors or categories.The training resolution is (512,384) which may result in blurry deformation for smaller fonts (resolved at high resolutions).

768 resolution weights（maybe early March, if we get more stars）

我们在768分辨率上调优训练策略，增加模型分辨率，拓展衣服类别和模特类别，敬请期待。

We optimize the training strategy at 768 resolution, increase model resolution, and expand clothing and model categories. Stay informed with us!

Installation

Clone the repository

git clone https://github.com/chenshine1/oms-Diffusion.git

Create a conda environment and install the required packages

conda create -n oms-diffusion python==3.10
conda activate oms-diffusion
pip install torch==2.0.1 torchvision==0.15.2 numpy==1.25.1 diffusers==0.25.1 opencv-python==4.8.0  transformers==4.31.0 gradio==4.16.0 safetensors==0.3.1 controlnet-aux==0.0.6 accelerate-0.21.0

Inference

python demo

python inference.py --cloth_path[your cloth path] --model_path[your model path]

gradio demo

python gradio_generate.py --model_path[your model path]

Linrstudio / oms-Diffusion