NVDiffrast based implementation of Latent-Paint from Latent-NeRF
Use text to edit textures of a 3D model or create new ones - powered by Stable Diffusion
Creating texture with prompt "Goldfish"
Editing an existing texture with prompt "Air Jordan 1"
Use the following Google Colab notebook to get started - creating a texture map for 3d car model using text prompt "Lightning McQueen" (~5 mins)
git clone https://github.com/NasirKhalid24/Latent-Paint-Mesh.git
cd Latent-Paint-Mesh
pip install -r requirements.txt
# Create a TOKEN file with your huggingface token from https://huggingface.co/settings/tokens
echo -n YOUR_TOKEN_HERE >> TOKEN
python main.py --mesh data/blub/blub_triangulated.obj --text "Goldfish" --epochs 2500 --log 250
optional arguments:
--gpu GPU GPU index (default 0)
--mesh MESH Path to .obj must have uv unwrapped
--material MATERIAL Path to starting material, if none will randomly initialize
--text TEXT Text prompt
--lr LR Learning rate
--scale SCALE Factor by which to scale up 64x64 texture map (default 2)
--log LOG Log every x epochs, set to -1 to disable logging (much faster)
--dist_min DIST_MIN Minimum camera distance
--dist_max DIST_MAX Maximum camera distance
--azim_min AZIM_MIN Minimum camera azimuth in degrees
--azim_max AZIM_MAX Maximum camera azimuth in degrees
--elev_min ELEV_MIN Minimum camera elevation in degrees wrt x plane (use -x and +x for elevation)
--elev_max ELEV_MAX Maximum camera elevation in degrees wrt x plane (use -x and +x for elevation)
--fov_min FOV_MIN Minimum FOV in degrees
--fov_max FOV_MAX Maximum FOV in degrees
--offset Offset mesh from center
--colab Enable for google colab logging
--prompt_aug Prompt augmentatio (side, back)
--auto_cam Automatically calculate camera params
--epochs EPOCHS How many iterations to run for
--guidance GUIDANCE Guidance scale for cfg (0 to 1)