What is the cost for training LADD?
HoiM opened this issue · comments
Thank you for your excellent work!
We are currently trying to apply LADD (Latent Adversarial Diffusion Distillation) on our own 2B text-to-image model.
Can you provide any more experimental details, especially the training cost, e.g. GPU hours, which kind of GPU, global batch size, etc. ?
Appreciate it!
We have a working implementation and training code for Latent Adversarial Diffusion Distillation, and achieved good results with multiple base models and resolutions: https://github.com/AMD-AIG-AIMA/AMD-Diffusion-Distillation. In our experiments we found that we can get reasonable results in 1 day on 8x AMD Instinct MI250 GPUs
Based on our experiments, training with 10k iterations with global batch size being 8 can roughly yield a reasonable result. We trained LADD with 8 A100 (with torch FSDP) and our model is a 2B text-to-image model.