raymondlo84 / openvino-ai-plugins-gimp

GIMP AI plugins with OpenVino Backend

OpenVINO™ AI Plugins for GIMP

This branch is under development.
Dedicated for GIMP 3, Python 3 and OpenVino.
⭐ ⭐ ⭐ ⭐ are welcome.

Current list of plugins:

[1] Super-Resolution
[2] Style-Transfer
[3] Inpainting
[4] Semantic-Segmentation
[5] Stable-Diffusion

Objectives

[1] Provides a set of OpenVino based plugins that add AI features to GIMP.
[2] Serve as a refrence code for how to make use of OpenVino in GIMP application for inferencing on Intel's' CPU & GPU
[3] Add AI to routine image editing workflows.

Contribution

Welcome people interested in contribution !! Please raise a PR for any new features, modifactions or bug fixes.

Use with GIMP

Installation Steps

1. Install GIMP 2.99.10 (revision 2) (Only windows and linux)
   
2. Install OpenVino Runtime 2022.3 https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/download.html .
   
3. Clone this repository: git clone https://github.com/intel/openvino-ai-plugins-gimp.git
   
4. windows install:
   openvino-ai-plugins-gimp\install.bat
   
5. Follow steps that are printed in terminal or cmd to add the gimpenv3 path to the GIMP GUI [Edit-> Preferences-> Folders-> Plugins].
   
6. Copy the weights folder to C:\Users\<user_name>\openvino-ai-plugins-gimp\weights
   
7. Download Stable-Diffusion models from https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino/tree/main and place it in C:\Users\<user_name>\openvino-ai-plugins-gimp\weights\stable-diffusion-ov
   
8. Download the clip-vit-large-patch14 tokenizer files - merges.txt, special_tokens_map.json, tokenizer_config.json, vocab.json from https://huggingface.co/openai/clip-vit-large-patch14/tree/main and place it in C:\Users\<user_name>\openvino-ai-plugins-gimp\weights\stable-diffusion-ov
   

Running GIMP

1. In a new command window run setupvars.bat from OpenVino toolkit folder.
   
2. Navigate to the GIMP installation directory:
   cd C:\Program Files\GIMP 2.99\bin
3. Start the GIMP application:
   gimp-2.99.exe
   
4. GIMP GUI should start up. Check if the OpenVino Plugins are loaded by going to layers -> OpenVino-AI-Plugins.
   

OpenVINO™ Semantic Segmentation Plugin

OpenVINO™ Super Resolution Plugin

OpenVINO™ Style Transfer Plugin

OpenVINO™ Inpainting Plugin

1. Open an image in GIMP.
   
2. Make sure there is alpha channel added to the image by right clicking on the image from layer section and selecting “Add alpha channel”
   
3. Add a new transparent layer of the same size as original image.
   
4. Select paint brush with white foreground color and black background color. Choose the thickness of the brush
   
5. Now paint the object that you want to remove from the image.
   
6. Select the new layer and image at the same. You should see “two items selected in layer section”
   

OpenVINO™ Image Generator Plugin with Stable Diffusion

1. Create a new layer of size 512x512
   
2. Select Stable Diffusion from the drop down list in layers -> OpenVINO-AI-Plugins
   
3. Enter a prompt, other parameters and select the device - CPU or GPU
   
4. Click on “Run Inference”. Wait for the total inference steps to get completed. (Can be viewed in Gimp output window)
   
5. If create gif option is selected, please note that performance will reduce. The generated gif is located in below path. You can play it in GIMP by going to Filters -> Animations -> Playback
   C:\Users\<user_name>\openvino-ai-plugins-gimp\gif\stable_diffusion.gif
   

Acknowledgements

• Plugin architecture inspired from GIMP-ML - https://github.com/kritiksoman/GIMP-ML/tree/GIMP3-ML
• Stable Diffusion Engine - https://github.com/bes-dev/stable_diffusion.openvino

License

Apache 2.0

Disclaimer

Stable Diffusion’s data model is governed by the Creative ML Open Rail M license, which is not an open source license. https://github.com/CompVis/stable-diffusion. Users are responsible for their own assessment whether their proposed use of the project code and model would be governed by and permissible under this license.