Giters

thewayur / AVeryComfyNerd

ComfyUI related stuff and things

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Overview

A variety of ComfyUI related stuff. You'll need different models and custom nodes for each different workflow.

Troubleshooting

When troubleshooting (working to fix issues) - such as with your local custom node installs, it's best to do all of these steps until resolution.

  • Make sure you've installed the drivers for your graphics card
  • These workflows require ComfyUI to run, so you'll need to install that first. See https://youtu.be/2r3uM_b3zA8 for an install guide
  • Install ComfyUI Manager next - https://github.com/ltdrdata/ComfyUI-Manager
  • Need a model or checkpoint? See the resources section below!
  • By default, models are saved in subdirectories under ComfyUI/models, though some custom nodes have their own models directory.
  • Don't mix SDXL and SD1.5 models (unless stated, such as SDXL needing the SD 1.5 vision model) - chances are you'll get an error!
  • Don't try to use SDXL models in workflows not designed for SDXL - chances are they won't work!
  • These are just workflows - no custom nodes here, so no code to go wrong :) Scroll down to see the table of workflows and their descriptions.
  • Some workflows are large. Zoom out to see more of the canvas.
  • Be sure to keep ComfyUI updated regularly - including all custom nodes. Old versions may result in errors appearing. This is the most common issue, so update now!
  • Use ComfyUI Manager to install missing custom nodes by clicking "Install Missing Custom Nodes"
  • If ComfyUI Manager can't find a node automatically, use the search feature
  • Custom node still red after installing it? Remember to restart ComfyUI!
  • Custom node still giving an error? Check the GitHub page for that custom node - maybe someone else has a similar issue open?
  • Not sure where the GitHub page is for a custom node? You can click on it via ComfyUI Manager
  • Check the output when ComfyUI starts up as issues can show up there
  • Try updating custom nodes manually (git pull)
  • Sometimes custom nodes make breaking changes and the same node may need to be re-created. Typically the custom node GitHub page has such information.
  • Sometimes custom nodes just break! Check the github page for the custom node causing any issues for more information and to raise issues
  • Sometimes custom nodes change functionality, so check for updates. Changes include:
    • Segement anything - mask output inverted & now returns multiple images
    • Dynamic Thresholding - output different
    • The original IPAdapter is becoming deprecated
  • The Microsoft Windows portable version of ComfyUI apparently has issues with various custom nodes, whereas normal installs are OK. Unknown error? Try a normal install!
  • Need more help? See this Playlist with loads of ComfyUI guides

Resources

Item Description Link
ComfyUI The main thing you'll need! https://github.com/comfyanonymous/ComfyUI
See https://youtu.be/2r3uM_b3zA8 for an install guide
ComfyUI Manager Install any missing nodes using this https://github.com/ltdrdata/ComfyUI-Manager
Stability AI Models & VAEs https://huggingface.co/stabilityai
Text-to-Image models Text-2-image models https://huggingface.co/models?pipeline_tag=text-to-image&sort=trending
SSD-1B Text2-image model https://huggingface.co/segmind/SSD-1B
ControlNet Models ControlNet Models https://huggingface.co/lllyasviel/sd_control_collection/tree/main
https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main
QR Code Monster Control Net ControlNet Model https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster
IP Adpater Github Repo https://github.com/tencent-ailab/IP-Adapter
IP Adapter models Models https://huggingface.co/h94/IP-Adapter
T2I Adapter Github Repo https://github.com/TencentARC/T2I-Adapter
Control LoRA Control Models https://huggingface.co/stabilityai/control-lora
AnimateDiff Original repo, many links and more info https://github.com/guoyww/AnimateDiff
Latent Consistency Models Models https://huggingface.co/latent-consistency
Upscale Wiki Many models & info https://upscale.wiki/wiki/Main_Page
Artist Style Studies SDXL Prompt output examples for inspiration https://sdxl.parrotzone.art/

Updating / Installing Custom Nodes

  • Use ComfyUI Manager to install and update custom nodes with ease!
  • Click "Install Missing Custom Nodes" to install any red nodes
  • Use the "search" feature to find any nodes
  • Be sure to keep ComfyUI updated regularly - including all custom nodes. Old versions may result in errors appearing.

Workflows available

In ComfyUI the image IS the workflow. Simply drag or load a workflow image into ComfyUI!

Workflow Description Version
Basic SDXL ControlNet workflow.
Introductory SDXL Canny & Depth ControlNet example.
See https://youtu.be/reqamcrPYiM for more information.		 SDXL
Basic QR Code Monster SD 1.5 ControlNet - make spiral art!
See also - https://youtu.be/D4oJz0w36ps		 SD 1.5
QR Code Monster SD 1.5 ControlNet - make animated spiral art!
See also: https://youtu.be/D4oJz0w36ps		 SD 1.5
Updated QR Code Monster SD 1.5 ControlNet with AnimateDiff and FreeU
Works best with the v1 QR Code Monster - https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster		 SD 1.5
AnimateDiff with Montion LoRA example. Pan up, down, left right, etc. SD 1.5
Instant LoRA 1
Inspired by AloeVera (almost identical).
Really simple, no training, "LoRA" like functionality.
SD 1.5. IP Adapter models:
1. https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus_sd15.bin -> custom_nodes/IPAdapter-ComfyUI/models.
2. https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> models/clip_vision.
Video guide - https://youtu.be/HtmIC6fqsMQ		 SD 1.5
Instant LoRA 2
As above, but with ControlNet to guide the shape		 SD 1.5
Instant LoRA 3
As above, but with QR Code Monster ControlNet too :)		 SD 1.5
Instant LoRA 4
As above, but with basic upscaling		 SD 1.5
Instant LoRA 5
As above, but with more upscaling to 16k+		 SD 1.5
Instant LoRA 6
As above, but different upscaling to 16k+		 SD 1.5
Morphing AI videos of any length using AnimateDiff. SD 1.5. Includes IPAdapter & Upscaling. IP Adapter models:
1. https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus_sd15.bin -> custom_nodes/IPAdapter-ComfyUI/models.
2. https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> models/clip_vision.
Video guide - https://youtu.be/6A3a0QNPhIs		 SD 1.5
Morphing AI videos of any length using AnimateDiff. SD 1.5. Includes Upscaling. Like above, but without IPAdapter controls. SD 1.5
SDXL "Instant LoRA" - basic.
Really simple, no training, "LoRA" like functionality.
Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter
Video - https://youtu.be/dGL02W4QatI		 SDXL
SDXL "Instant LoRA" - with CLIP Vision
Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter
Also use "Revisions" CLIP vision - https://huggingface.co/stabilityai/control-lora		 SDXL
SDXL "Instant LoRA" - with CLIP Vision & ControlNet
Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter
Also use "Revisions" CLIP vision - https://huggingface.co/stabilityai/control-lora		 SDXL
AnimateDiff + QRCode (Vid2Vid)
Use any high-contrast input video to create guided animations! Spirals away...		 SD 1.5

SD 1.5 Reposer (2 versions) - single face image to any pose. Get consistent faces!
No "roop" or similar face-swapping nodes required = easy install!
SD 1.5 ControlNet models:
https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main
IP Adapter models:
1. https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus-face_sd15.bin
2. https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors
NOTE Now uses the more updated version of IPAdapter - IPAdapter_Plus!
Reposer & Reposer Plus Bypass Edition
Original Reposer Basic Video guide - https://youtu.be/SacK9tMVNUA
Original Reposer Plus Video guide - https://youtu.be/ZcCfwTkYSz8		 SD 1.5
SD 1.5 Video Styler! Combining IPAdapter with Video-to-video for strange styles and weird animations
Uses https://github.com/cubiq/ComfyUI_IPAdapter_plus
The pre-trained models are available on huggingface, download and place them in the ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models directory.
For SD1.5 you need:
* ip-adapter_sd15.bin
* ip-adapter_sd15_light.bin
* ip-adapter-plus_sd15.bin
* ip-adapter-plus-face_sd15.bin
Additionally you need the image encoder to be placed in the ComfyUI/models/clip_vision/ directory.
They are the same models used by the other IPAdapter custom nodes ;) - symlinks are your friend!
Video guide - https://youtu.be/kJp8JzA2aVU		 SD 1.5
SDXL version of Reposer using the SDXL "IPAdapter Plus Face" model
Pick a face then add a body in any pose - no training!
Works with photorealistic faces, anime faces, cartoon faces, etc		 SDXL
SSD-1B Workflow - SDXL for 8GB VRAM cards!
Model - https://huggingface.co/segmind/SSD-1B
Video - https://youtu.be/F-bKndyQ7L8		 SSD-1B
LCM LoRA vs Normal 1.5, SDXL, SSD-1B
IPAdapter Attention Masking Example
Video https://youtu.be/riLmjBlywcg		 1.5
IPAdapter Attention Masking Example with extra toppings (LCM, Facefix)
Video https://youtu.be/riLmjBlywcg		 1.5
Stable Video Diffusion example with a simple upscale and frame interpolation SVD
SDXL Turbo - 1 step diffusion! SDXL Turbo, SD2 Turbo
A very basic attempt at a "Comfy MagicAnimate". Needs more work :)
Links:
Magic Animate - https://github.com/magic-research/magic-animate
Magic Animate (Windows) - https://github.com/sdbds/magic-animate-for-windows
DreamingTulpa - https://twitter.com/dreamingtulpa/status/1730876691755450572
CocktailPeanut - https://twitter.com/cocktailpeanut/status/1732052909720797524
Google Colab - https://github.com/camenduru/MagicAnimate-colab
Huggingface Space - https://huggingface.co/spaces/zcxu-eric/magicanimate
Vid2DensePose - https://github.com/Flode-Labs/vid2densepose

Model Downloads for the MagicAnimate Gradio App:
mkdir -p magic-animate/pretrained_models
cd magic-animate/pretrained_models
git lfs clone https://huggingface.co/runwayml/stable-diffusion-v1-5 -b fp16
git lfs clone https://huggingface.co/stabilityai/sd-vae-ft-mse
git lfs clone https://huggingface.co/zcxu-eric/MagicAnimate
Video - https://youtu.be/td27SyA9M80		 SD 1.5
Steerable Motion - Image Batch with AnimateDiff SD 1.5

Custom Node List

The current custom node list is shown in the image below. This is a complete list. Not all nodes are used in workflows.
Install custom nodes using ComfyUI manager



About

ComfyUI related stuff and things

License:MIT License