jtydhr88 / sd-3dmodel-loader

A custom extension for stable diffusion webui to load local 3D model/animation

Stable Diffusion WebUI 3D Model&Pose Loader

A custom extension for AUTOMATIC1111/stable-diffusion-webui that allows you to load your local 3D model/animation inside webui, or edit pose as well, then send screenshot to txt2img or img2img as your ControlNet's reference image.

Installation

Just like you install other extension of webui:

1. go to Extensions -> Install from URL
2. paste this repo link
3. install
4. go to Installed, apply and restart UI

Or you can install from Extensions -> Available, and load the official extension link, then you can find this extension on the list.

Quick Start

1. click Load Model button, select one model from your local, check here for support formats,
2. or click Load Hand Model/Load Body Model to load embedded hand/body model.
3. On the main canvas, you can use:
   1. Mouse left button -> rotate the camera
   2. Mouse wheel -> zoom in or out
   3. Mouse right button -> move the camera
4. Send the screenshot to ControlNet

More Details and features

• Support Formats
• Settings
• Settings From Page
• Render Mode
• Body model and pose edit
• Hand Model and gesture edit
• Send to
• Local Dev

Gradio.Model3D?

I know gradio has its own 3D model component called Gradio.Model3D, but it only supports three formats: obj, glb and gltf, I think it is not enough, at least it should support FBX animation, so I build my own extension.

Credits

Created by jtydhr88 and special thanks to other contributors:

• missionfloyd
• chucktobbes
• Everyone who gives feedback

My other extension for Stable diffusion webui

• Canvas Editor A custom extension for AUTOMATIC1111/stable-diffusion-webui that integrated a full capability canvas editor which you can use layer, text, image, elements and so on, then send to ControlNet, basing on Polotno.
• StableStudio Adapter A custom extension for AUTOMATIC1111/stable-diffusion-webui to extend rest APIs to do some local operations, using in StableStudio.
• Txt/Img to 3D Model A custom extension for sd-webui that allow you to generate 3D model from txt or image, basing on OpenAI Shap-E.
• 3D Editor A custom extension for sd-webui that with 3D modeling features (add/edit basic elements, load your custom model, modify scene and so on), then send screenshot to txt2img or img2img as your ControlNet's reference image, basing on ThreeJS editor.