Exploring the world of stable difussion and LLMs
- the
code-workspace
file in this repo contains some bootscraps that assumes you have- a conda environment called
pytorch
. This can be easily modified to any other name. - that you are using bash (Git Bash or GNU Bash should both work)
- a conda environment called
- The steps in the repo's readme matches what I did closely in creating the conda environment. Briefly as below:
- Run the commands below
# Create environment conda create -n pytorch python=3.10.6 # Activate environment conda active pytorch # Start local webserver ./webui-user.bat # Wait for "Running on local URL: http://127.0.0.1:7860" and open that URI.
- note that
webui-user.bat
will create a python virtual environment despite dedicated conda environment is created. If you don't prefer this to happen perform what the script does manually.- or don't use conda and directly use your system python.
- Download the stable diffusion models from here for v2.1 and here for v1.4.
- Need use additional arg of
--xformers
and--medvram
if GPU VRAM is less than 12GB. After using these arg web server is successfully launched even using GPU of only 2GB VRAM. - HOWEVER, there might still be error when trying to generate image using GPU with less VRAM (i.e. 2GB)
- References
- After following the official setup guide, tried to run VideoControl model
- Got a warning of symlink not enabled and will result in more disk space use. Two solutions
- Run as admin
- Enable Developer mode in windows (went for this)
- Got another error when trying to run model,
AssertionError: Torch not compiled with CUDA enabled
- Tried install torch with cuda
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia
- Found that the solver is too slow and upgraded conda to use libmamba solver
- Update conda version forcefully using conda install conda={latest conda version} because of this exact error
- Tried install torch with cuda
- Need to install chardet after getting
ModuleNotFoundError: No module named 'chardet'
errorpip install chardet
- However, despite the python command suceed in this stage, ran into the issue of insufficient GPU RAM as stated in the readme of the project
- Minimum of 7GB VRAM is needed, my machine had 6GB 😰
- Updated CUDA version from 11.6 to 12.1. Pending retrying all the steps again.
- Steps
conda create -n lvdm python=3.8.5
conda activate lvdm
pip install -r requirements_xformer.txt
- Overwrite pip install of pytorch with pytorch compiled with CUDA
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install chardet
- Lastly the steps from the readme for running
VideoControl
PROMPT="An ostrich walking in the desert, photorealistic, 4k" VIDEO="input/flamingo.mp4" OUTDIR="results/" NAME="video_adapter" CONFIG_PATH="models/adapter_t2v_depth/model_config.yaml" BASE_PATH="models/base_t2v/model.ckpt" ADAPTER_PATH="models/adapter_t2v_depth/adapter.pth" python scripts/sample_text2video_adapter.py \ --seed 123 \ --ckpt_path $BASE_PATH \ --adapter_ckpt $ADAPTER_PATH \ --base $CONFIG_PATH \ --savedir $OUTDIR/$NAME \ --bs 1 --height 256 --width 256 \ --frame_stride -1 \ --unconditional_guidance_scale 15.0 \ --ddim_steps 50 \ --ddim_eta 1.0 \ --prompt "$PROMPT" \ --video $VIDEO
- Conclusions
- Still hitting the VRAM insufficient issue. Have to put off running any models that requires more than 6GB GPU memory locally. ☠️
- Steps