xizzat / Fooocus-Extra

Merge all interesting forks for Fooocus

Fooocus - mashb1t's 1-Up Edition

The purpose of this fork is to add new features / fix bugs and contribute back to Fooocus.

As a collaborator & contributor of the Fooocus repository you can find me in almost every issue, pull request, discussion etc.

Sadly the creator of Fooocus has gone dark multiple times for an extended amount of time, which is why I took matters into my own hands.

Additional features included in this fork:

(mostly a reflection of my PRs)

• ✨ lllyasviel#958 - NSFW image censoring (config and UI)
• 🐛 lllyasviel#981 - prevent users from skipping/stopping other users tasks in queue (multi-user capabilities) + rework advanced_parameters (removal + PID handling)
• ✨ lllyasviel#985 - add list of 100 animals to wildcards
• ✨ lllyasviel#1013 - add advanced parameter for disable_intermediate_results (progress_gallery, prevents UI lag when generation is too fast)
• ✨ lllyasviel#1039 - add prompt translation
• ✨ lllyasviel#1043 - add lcm realtime canvas painting (not merged to main in this repository)
• ✨ lllyasviel#1167 - update model BluePencil XL v0.5 to v3.1.0
• ✨ lllyasviel#1570 - add preset selection to Gradio UI (session based)
• 🐛 lllyasviel#1578 - add workaround for changing prompt while generating
• ✨ lllyasviel#1580 - add preset for SDXL Turbo (model DreamShaperXL_Turbo)
• ✨ lllyasviel#1616 - add config setting for default_max_image_number
• 🐛 lllyasviel#1668 - fix path_outputs directory creation if it doesn't exist
• ✨ show more details for each performance setting, e.g. steps
• ✨ add default_overwrite_step handling for meta data and gradio (allows turbo preset switching to set default_overwrite_step correctly)
• ✨ lllyasviel#1762 - add style preview on mouseover
• 🐛 lllyasviel#1784 - correctly sort files, display deepest directory level first
• ✨ lllyasviel#1785 - update model Juggernaut XL v6 to v8
• ✨ lllyasviel#1809 - reduce file size of preview images
• ✨ lllyasviel#1932 - use consistent file name in gradio
• ✨ lllyasviel#1863 - image extension support (png, jpg, webp)
• ✨ lllyasviel#1938 - automatically describe image on uov image upload if prompt is empty
• ✨ lllyasviel#1940 - meta data handling, schemes: Fooocus (json) and A1111 (plain text). Compatible with Civitai.
• ✨ lllyasviel#1979 - prevent outdated history log link after midnight
• ✨ lllyasviel#2032 - add inpaint mask generation functionality using rembg, incl. segmentation support

✨ = new feature
🐛 = bugfix
abc = merged

---

Feature showcase

lllyasviel#1570 - Preset Selection

No need to restart your browser to change a preset ever again. Combined with total user isolation, every user can now set and use any preset they desire. You can even reload your presets in the browser if you've changed them.

---

lllyasviel#2032 - Automated Mask Generation + Mask Prompting

299776543-696b97e8-bc05-4d52-86f6-0f1693a7dd25.mp4

Videos by @rayronvictor

Mask generation by cloth category

299493287-5a030d4e-280e-46cb-a8b1-50d264a70d2d.mp4

Mask generation by prompt

299776543-696b97e8-bc05-4d52-86f6-0f1693a7dd25.mp4

---

lllyasviel#1940 - Metadata Handling - Compatible with Civitai & A1111

This feature offers activatable metadata persistency in images for both a Fooocus (json) and A1111 (plain text) meta data scheme, where the latter is 100% compatible with A1111 and Civitai, but can not be used to reproduce the image outside of Fooocus, as there are so many improvements and special things happening in Fooocus it's just not applicable anywhere else.

• Supports metadata for PNG (PngInfo) + JPG and WebP (both EXIF).
• Save & restore configurations directly from images
• You can also configure a copyright / creator tag

Gradio (setting in Developer Debug Mode)

Default is Fooocus Scheme

Config options

"default_save_metadata_to_images": true,
"default_metadata_scheme": "a1111",
"metadata_created_by": "mashb1t"

Arg --disable-metadata

--disable-metadata completely prevents metadata processing and output in Gradio

Metadata Reader

1. open Image Input > Metadata tab
2. drag & Drop image to image upload
3. automatic preview of image metadata
4. apply metadata to Gradio inputs on button click

Fooocus scheme

A1111 scheme

Metadata in files

Speed Fooocus scheme

LCM A1111 scheme (yes, with negative prompt, because it technically exists but doesn't have an influence)

Speed A1111 scheme

Civitai

Speed Fooocus scheme

LCM A1111 scheme

Speed A1111 scheme

---

This is a fork of Fooocus. This fork integrates the following:

• Insightface/inswapper library used by roop, ReActor, and others
• PhotoMaker based on 🤗 diffusers
• InstantID based on 🤗 diffusers

The goal of this repository is to stay up-to-date with the main repository, while also maintaining the above integrations.

For more detailed and official documentation, please refer to lllyasviel's repository.

A standalone installation does not exist for this repository.

Installation (Windows)

The installation assumes CUDA 11.8. If you need a different version, please update configure.bat with the correct URL to the desired CUDA version.

1. Run git clone https://github.com/machineminded/Fooocus-inswapper.git
2. Execute configure.bat

Inswapper Usage

Inswapper will activate if "Input Image" and "Enabled" are both checked.

1. .\venv\Scripts\activate
2. python launch.py

usage.mp4

PhotoMaker Usage

In this fork, PhotoMaker utilizes 🤗 diffusers, so it runs outside of the ksampler pipelines. I'd like to eventually add inpainting and ControlNet for 🤗 diffusers but it will take some time. Keep in mind that PhotoMaker currently requires 15GB of VRAM! The following Fooocus configuration items are passed to the PhotoMaker 🤗 diffusers pipeline:

• Resolution (width and height)
• Prompt (and generated prompts from selected styles)
• Negative Prompt (and generated prompts from selected styles)
• Steps
• CFG/Guidance Scale
• Seed
• LoRAs
• Sampler (not fully implemented)
• Scheduler (not fully implemented)

PhotoMaker General Usage

1. Navigate to the PhotoMaker tab.
2. Click "Enable"
3. Load images from your PC.
4. Enter your prompt and be sure to include "man img" or "woman img" depending on the subject at hand. img in the prompt is expected by PhotoMaker.
5. Click "Generate"

Experiment with also adding an image to the Inswapper tab to overlay the generated image.

Note: Unchecking "Enable" will unload the PhotoMaker pipeline from memory!

PhotoMaker LoRA Usage

1. Select the LoRAs you want to use as usual.
2. Navigate to the PhotoMaker tab.
3. Click "Enable" then click "Generate"

If you change the LoRAs or their weights:

1. Uncheck "Enabled" to unload the model from memory
2. Re-check "Enabled" and click "Generate" to reload the LoRAs and the pipeline into memory.

Supported PhotoMaker samplers

• euler
• euler ancestral
• DPM++ 2M SDE
• DPM++ 2M SDE Karras
• Will default to DDIM Scheduler for anything else

InstantID Usage

>>> Click here to download <<<

In this fork, InstantID utilizes 🤗 diffusers, so it runs outside of the ksampler pipelines. I'd like to eventually add inpainting and ControlNet for 🤗 diffusers but it will take some time. This requires high amounts of VRAM (easily 18GB or more). The following Fooocus configuration items are passed to the InstantID 🤗 diffusers pipeline:

• Resolution (width and height)
• Prompt (and generated prompts from selected styles)
• Negative Prompt (and generated prompts from selected styles)
• Steps
• CFG/Guidance Scale
• Seed
• LoRAs
• Sampler (not fully implemented)
• Scheduler (not fully implemented)

InstantID General Usage

1. Navigate to the InstantID tab.
2. Click "Enable"
3. Load images from your PC.
4. Enter your prompt and be sure to include "man img" or "woman img" depending on the subject at hand. img in the prompt is expected by PhotoMaker.
5. Click "Generate"

Experiment with also adding an image to the Inswapper tab to overlay the generated image.

Note: Unchecking "Enable" will unload the InstantID pipeline from memory!

InstantID LoRA Usage

1. Select the LoRAs you want to use as usual.
2. Navigate to the InstantID tab.
3. Click "Enable" then click "Generate"

If you change the LoRAs or their weights:

1. Uncheck "Enabled" to unload the model from memory
2. Re-check "Enabled" and click "Generate" to reload the LoRAs and the pipeline into memory.

Supported InstantID samplers

• euler
• euler ancestral
• DPM++ 2M SDE
• DPM++ 2M SDE Karras
• Will default to DDIM Scheduler for anything else

Colab

Note that the minimal requirement is 4GB Nvidia GPU memory (4GB VRAM) and 8GB system memory (8GB RAM). This requires using Microsoft’s Virtual Swap technique, which is automatically enabled by your Windows installation in most cases, so you often do not need to do anything about it. However, if you are not sure, or if you manually turned it off (would anyone really do that?), or if you see any "RuntimeError: CPUAllocator", you can enable it here:

Click here to see the image instructions.

And make sure that you have at least 40GB free space on each drive if you still see "RuntimeError: CPUAllocator" !

Please open an issue if you use similar devices but still cannot achieve acceptable performances.

Note that the minimal requirement for different platforms is different.

See also the common problems and troubleshoots here.

Switching from Fooocus to this fork

1. open a terminal in your Fooocus folder (the one with your config.txt)

2. execute git status. You should see the following:

   On branch main
   Your branch is up to date with 'origin/main'.
   
   nothing to commit, working tree clean
   

   If not, execute git reset --hard origin/main and check git status again.

3. execute

   git remote set-url origin https://github.com/mashb1t/Fooocus.git
   git pull
   

4. activate your venv (not necessary when installed from 7z) and update your python packages depending on your environment (7z, venv, conda, etc.)

   Example for Windows (7z): ..\python_embeded\python.exe -m pip install -r "requirements_versions.txt"

5. start Fooocus either by opening the run.bat or corresponding entrypoint (same as before)

OR

Windows: download the 7z file, extract it and run run.bat. You may want to copy over already downloaded checkpoints / LoRAs / etc.

Colab

(Last tested - 2023 Dec 12)

Colab	Info
	Fooocus Official

Issues

Please report any issues in the Issues tab. I will try to help as much as I can.

To Do

1. 🚀 Allow changing of insightface parameters (Inswapper)
2. 🚀 Allow customizable target image (Inswapper)
3. 🚀 Increase diffusers pipeline to > 77 tokens (PhotoMaker)
4. 🚀 Allow dynamic loading of LoRAs into diffusers pipeline (PhotoMaker)