Runtime environment crushes after RAM usage goes over 12GB
volotat opened this issue · comments
Alexey Borsky commented
I tired to generate anything using this colab but each time it crushes after system RAM goes over 12GB. Is there any way to mitigate this problem via controlling number of frames generated or the size of the frame?
camenduru commented
Hi @volotat 👋 please try again maybe works and if modelscope developers add new features I will add we are using this pipeline and thanks ❤ for the https://github.com/volotat/Reference-based-SD-CN-Animation look super cool 🔥 I will try
camenduru commented
and maybe we can edit max_frames
in /content/models/configuration.json
{ "framework": "pytorch",
"task": "text-to-video-synthesis",
"model": {
"type": "latent-text-to-video-synthesis",
"model_args": {
"ckpt_clip": "open_clip_pytorch_model.bin",
"ckpt_unet": "text2video_pytorch_model.pth",
"ckpt_autoencoder": "VQGAN_autoencoder.pth",
"max_frames": 16,
"tiny_gpu": 1
},
"model_cfg": {
"unet_in_dim": 4,
"unet_dim": 320,
"unet_y_dim": 768,
"unet_context_dim": 1024,
"unet_out_dim": 4,
"unet_dim_mult": [1, 2, 4, 4],
"unet_num_heads": 8,
"unet_head_dim": 64,
"unet_res_blocks": 2,
"unet_attn_scales": [1, 0.5, 0.25],
"unet_dropout": 0.1,
"temporal_attention": "True",
"num_timesteps": 1000,
"mean_type": "eps",
"var_type": "fixed_small",
"loss_type": "mse"
}
},
"pipeline": {
"type": "latent-text-to-video-synthesis"
}
}
camenduru commented
please watch this https://www.youtube.com/watch?v=b8D4am73e6I
Alexey Borsky commented
Thank you a lot for a quick response and the video. It does work now.