cuda.Out Of Memory
yifenglv46 opened this issue · comments
i have 12G of vram,is it enough?
I don't know if it's an existing problem or something else
python app.py
#######################
Running in eps mode
#######################
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Load model from [pretrained/pfd/vae/sd-v2-0-base-autokl.pth] strict [True].
Load autoencoderkl with total 83653863 parameters,79145.299 parameter sum.
Load swin with total 195201204 parameters,44284.855 parameter sum.
Load seecoder_decoder with total 27783168 parameters,13733.044 parameter sum.
Load seecoder_query_transformer with total 71130624 parameters,20814.307 parameter sum.
Load seecoder with total 294114996 parameters,78832.206 parameter sum.
Load openai_unet_2d_next with total 859520964 parameters,100048.114 parameter sum.
Load controlnet with total 361279120 parameters,40656.570 parameter sum.
Load pfd_with_control with total 1598568943 parameters,298682.189 parameter sum.
Load context encoder from [pretrained/pfd/seecoder/seecoder-v1-0.safetensors] strict [True].
Load diffuser from [pretrained/pfd/diffuser/Deliberate-v2-0.safetensors] strict [True].
Load controlnet from [pretrained/controlnet/control_sd15_canny_slimmed.safetensors] strict [True].
###################
Running in FP16
###################
Caching examples at: '/home/lyf/Prompt-Free-Diffusion/gradio_cached_examples/26'
Data shape for DDIM sampling is [1, 4, 96, 128], eta 0.0
DDIM Sampler: 0%| | 0/50 [00:01<?, ?it/s]
Traceback (most recent call last):
File "/home/lyf/Prompt-Free-Diffusion/app.py", line 481, in
interface()
File "/home/lyf/Prompt-Free-Diffusion/app.py", line 418, in interface
gr.Examples(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/gradio/helpers.py", line 69, in create_examples
utils.synchronize_async(examples_obj.create)
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/gradio/utils.py", line 409, in synchronize_async
return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs)
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/fsspec/asyn.py", line 96, in sync
raise return_result
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
result[0] = await coro
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/gradio/helpers.py", line 273, in create
await self.cache()
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/gradio/helpers.py", line 308, in cache
prediction = await Context.root_block.process_api(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/gradio/blocks.py", line 1012, in process_api
result = await self.call_function(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/gradio/blocks.py", line 830, in call_function
prediction = await anyio.to_thread.run_sync(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/home/lyf/Prompt-Free-Diffusion/app.py", line 262, in action_inference
x, _ = sampler.sample(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/ddim.py", line 72, in sample
samples, intermediates = self.ddim_sampling(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/ddim.py", line 116, in ddim_sampling
outs = self.p_sample_ddim(
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/ddim.py", line 150, in p_sample_ddim
e_t_uncond, e_t = self.model.apply_model(x_info, t_in, c_info).chunk(2)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/pfd.py", line 473, in apply_model
ccs = self.ctl(x, hint=cc, timesteps=timesteps, context=c)
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/controlnet.py", line 318, in forward
h = module(h, emb, context)
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/openaimodel.py", line 83, in forward
x = layer(x, context)
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/attention.py", line 365, in forward
x = block(x, context=context[i])
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/attention.py", line 300, in forward
return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/diffusion_utils.py", line 96, in checkpoint
return CheckpointFunction.apply(func, len(inputs), *args)
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/autograd/function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/diffusion_utils.py", line 108, in forward
output_tensors = ctx.run_function(*ctx.input_tensors)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/attention.py", line 303, in _forward
x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
File "/home/lyf/miniconda3/envs/prompt-free-diffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/lyf/Prompt-Free-Diffusion/lib/model_zoo/attention.py", line 188, in forward
sim = einsum('b i d, b j d -> b i j', q, k) * self.scale
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 4.50 GiB (GPU 0; 12.00 GiB total capacity; 7.66 GiB already allocated; 1.96 GiB free; 7.91 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
(prompt-free-diffusion) lyf@PC-20170106ZWYY:~/Prompt-Free-Diffusion$
Unfortunately, I think this is just the OOM error, 768x1024 images requires a lot of memory, please try images with smaller size.
for me, "cache_examples = False", and it works