SooLab / Free-Bloom

[NeurIPS 2023] Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator

Free-Bloom

This repository is the official implementation of Free-Bloom.

Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator

Hanzhuo Huang∗ , Yufan Feng∗, Cheng Shi, Lan Xu, Jingyi Yu, Sibei Yang†

*Equal contribution; †Corresponding Author

Setup

Requirements

conda create -n fb python=3.10
conda activate fb
pip install -r requirements.txt

Installing xformers is highly recommended for more efficiency and speed on GPUs. To enable xformers, set enable_xformers_memory_efficient_attention=True (default).

Usage

Generate

python main.py --config configs/flowers.yaml

Change the path of diffusion models to your own for the pretrained_model_path key in config yaml file.

Results

A Flower is blooming

Volcano eruption

A rainbow is forming

Citation

@article{freebloom,
	title={Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator},
	author={Huang, Hanzhuo and Feng, Yufan and Shi, Cheng and Xu, Lan and Yu, Jingyi and Yang, Sibei},
	journal={arXiv preprint arXiv:2309.14494},
	year={2023}
}