liamzebedee / 3d-headshot-pipeline

A text-to-3D headshot pipeline for an AI TV show

LLama69's 3D HeadGen

A text-to-3D asset pipeline I built for the AI Rick and Morty TV show to generate 3D characters.

1. Generates 4x candidate images (.png's) in parallel using Stable Diffusion XL, with prompt/antiprompt/reference image configured.
2. Feeds these 4x images to a DreamGaussian model, generating 4x 3D models in parallel (.obj, .mtl)
3. Downloads .zip containing 3D model (.obj) and textures (.mtl, .png)
4. Live imports into Unity and rigs them onto a generic human model https://github.com/liamzebedee/unity-runtime-obj-importer and custom code.

See docs/pipeline.md for more info.

Usage.

Sign up for the Replicate service.

export REPLICATE_API_KEY=...
pipenv install
python headshot/server.py

# Then run an inference for "mario".
curl -X POST -H "Content-Type: application/json" -d '{"character": "mario"}' http://0.0.0.0:10001/v1/character-heads/generate

Demos.

A 3D donald trump generated at runtime:

Sam altman telling codebullet about AGI

Gordon Ramsey and Dutch guys:

Internals.

Filesystem layout:

The boys:

Notice.

Copyright LLaMa69 Inc, 2023.