Mystfit / Unreal-StableDiffusionTools

Animate Unreal Engine scenes using Stable Diffusion

Stable Diffusion Tools for Unreal Engine

Description

This plugin generates AI synthesized images in Unreal Editor.

Join us on Discord.

Requirements

• Unreal Engine 5.1+
• CUDA compatible GPU (Nvidia)
• Minimum 6GB Vram to run Unreal and half-precision floating point (fp16) Stable Diffusion simultaneously
• Git

Installation

Setup instructions are available in the wiki.

Usage

Video walkthrough for v0.4

Compatible generators

This plugin provides three different AI image generator systems (bridges) out of the box. Local generation using the Diffusers library, remote generation using the Stability.AI SDK and Dream Studio and remote generation using Stable Horde.

FAQ

• Help, I don't know how any of this works.
  • Take a look at the wiki.

Known issues

• Sequencer generation options section isn't serialising pipeline assets and stages.
• Cancelling a model load can occasionally disable the generation button. Relaunch the editor to fix.

Roadmap

• Serialisable fields for the plugin UI.
• Queue multiple image generations.
• Paint tools for mask editing (inpaint).
• Better documentation.