Shipeng Zhu, Pengfei Fang, Chenjie Zhu, Zuoyan Zhao, Qiang Xu, Hui Xue
Paper: (arXiv 2401.14832), (AAAI-24)
This repository offers the official Pytorch code for this paper. If you have any questions, feel free to contact Shipeng Zhu (shipengzhu@seu.edu.cn) or Chenjie Zhu (chenjiezhu@seu.edu.cn).
-
Clone this repo
-
Create a conda environment and activate it.
-
Install related version Pytorch following
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia -
Install the required packages
-
Download the pre-trained checkpoints, and and move these files into the "checkpoints".
python inference.py --config xx --input_dir input --output_dir output --save_sp False- config: The path loading yaml file.
- input_dir: The input image path.
- output_dir: The Output image path.
- save_sp: Whether to save structure prediction images.
- Download the TII-HT and TII-ST datasets from: Baidu Cloud, Passwd: h5i0
- Download the Checkpoints from: Baidu Cloud, Passwd: dlr6; Google Drive.
python train_spm.py- Modify the training configuration in this file ——"config/train_spm.yaml"
python train_rm.py
- Modify the training configuration in this file ——"config/train_rm.yaml"
- Note that training RM requires a pre-trained SPM checkpoint, and the path should be modified in the above file.
- Download the checkpoint of the pre-trained CRNN model into the path: "crnn/data/"
@inproceedings{zhu2024gsdm,
title={Text image inpainting via global structure-guided diffusion models},
author={Zhu, Shipeng and Fang, Pengfei and Zhu, Chenjie and Zhao, Zuoyan and Xu, Qiang and Xue, Hui},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={38},
number={7},
pages={7775-7783},
year={2024}
}