muhammad-ahmed-ghani / PowerPaint

A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting

Project Page | Paper | Online Demo(OpenXlab)

This README provides a step-by-step guide to download the repository, set up the required virtual environment named "PowerPaint" using conda, and run PowerPaint with or without ControlNet.

Feel free to try it and give it a star!⭐

🚀 News

April 7, 2024🔥

• We open source the model weights and code for PowerPaint v2.

April 6, 2024🔥

• We have retrained a new PowerPaint, taking inspiration from Brushnet. The Online Demo has been updated accordingly. We plan to release the model weights and code as open source in the next few days.
• Tips: We preserve the cross-attention layer that was deleted by BrushNet for the task prompts input.

	Object insertion	Object Removal	Shape-guided Object Insertion	Outpainting
Original Image
Output

December 22, 2023🔧

• The logical error in loading ControlNet has been rectified. The gradio_PowerPaint.py file and Online Demo have also been updated.

December 18, 2023

Enhanced PowerPaint Model

• We are delighted to announce the release of more stable model weights. These refined weights can now be accessed on Hugging Face. The gradio_PowerPaint.py file and Online Demo have also been updated as part of this release.

---

Installation

conda create -n powerpaint python=3.10 -y
conda activate powerpaint
conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia
pip install openmim
pip install mmcv==2.1.0 -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1/index.html
pip install chardet
mim install mmengine
mim install mmagic
pip install peft
pip install -r requirements.txt

PowerPaint v2

python gradio_PowerPaint_BrushNet.py

This command will launch the Gradio interface for PowerPaint.

Feel free to explore and edit images with PowerPaint!

BibTeX

@misc{zhuang2023task,
      title={A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting},
      author={Junhao Zhuang and Yanhong Zeng and Wenran Liu and Chun Yuan and Kai Chen},
      year={2023},
      eprint={2312.03594},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}