Openpose Editor for ControlNet in Stable Diffusion WebUI
This extension is specifically build to be integrated into Stable Diffusion WebUI's ControlNet extension.
Translations of README.md
Prerequisite
ControlNet 1.1.216
+
Installation
On UI restart, the extension will try to download the compiled Vue app from
Github. Check whether stable-diffusion-webui\extensions\sd-webui-openpose-editor\dist
exists and has content in it.
Some users in China have reported having issue downloading dist with the autoupdate script. In such situtations, the user has 2 following options to get dist manually:
Option1: Build the application
Make sure you have nodeJS environment ready and follow Development
section.
Run npm run build
to compile the application.
Option2: Download the compiled application
You can download the compiled application(dist.zip
) from the
release page.
Unzip the package in the repository root and make sure hte unziped directory is
named dist
.
Usage
The openpose editor core is build with Vue3. The gradio extension script is
a thin wrapper that mounts the Vue3 Application on /openpose_editor_index
.
The user can directly access the editor at localhost:7860/openpose_editor_index
if desired, but the main entry point is invoking the editor in the ControlNet
extension. In ControlNet extension, select any openpose preprocessor, and hit
the run preprocessor button. A preprocessor result preview will be genereated.
Click Edit
button at the bottom right corner of the generated image will bring
up the openpose editor in a modal. After the edit, clicking the
Send pose to ControlNet
button will send back the pose to ControlNet.
Following demo shows the basic workflow:
Features
- Support for face/hand used in controlnet.
- The extension recognizes the face/hand objects in the controlnet preprocess results.
- The user can add face/hand if the preprocessor result misses them. It can
be done by either
- Add Default hand (Face is not supported as face has too many keypoints (70 keypoints), which makes adjust them manually really hard.)
- Add the object by uploading a pose JSON. The corresponding object of the first person will be used.
- Visibility toggle
- If a keypoint is not recognized by ControlNet preprocessor, it will have
(-1, -1)
as coordinates. Such invalid keypoints will be set as invisible in the editor. - If the user sets a keypoint as invisible and send the pose back to controlnet, the limb segments that the keypoint connects will not be rendered. Effectively this is how you remove a limb segment in the editor.
- If a keypoint is not recognized by ControlNet preprocessor, it will have
- Group toggle
- If you don't want to accidentally select and modify the keypoint of an canvas object (hand/face/body). You can group them. The grouped object will act like it is a single object. You can scale, rotate, skew the group.
Development
Recommended IDE Setup
VSCode + Volar (and disable Vetur) + TypeScript Vue Plugin (Volar).
.vue
Imports in TS
Type Support for TypeScript cannot handle type information for .vue
imports by default, so we replace the tsc
CLI with vue-tsc
for type checking. In editors, we need TypeScript Vue Plugin (Volar) to make the TypeScript language service aware of .vue
types.
If the standalone TypeScript plugin doesn't feel fast enough to you, Volar has also implemented a Take Over Mode that is more performant. You can enable it by the following steps:
- Disable the built-in TypeScript Extension
- Run
Extensions: Show Built-in Extensions
from VSCode's command palette - Find
TypeScript and JavaScript Language Features
, right click and selectDisable (Workspace)
- Run
- Reload the VSCode window by running
Developer: Reload Window
from the command palette.
Customize configuration
See Vite Configuration Reference.
Project Setup
npm install
Compile and Hot-Reload for Development
npm run dev
Type-Check, Compile and Minify for Production
npm run build
Vitest
Run Unit Tests withnpm run test:unit
ESLint
Lint withnpm run lint