This is the official repository of MotionDirector.
MotionDirector: Motion Customization of Text-to-Video Diffusion Models.
Rui Zhao,
Yuchao Gu,
Jay Zhangjie Wu,
David Junhao Zhang,
Jiawei Liu,
Weijia Wu,
Jussi Keppo,
Mike Zheng Shou
MotionDirector can customize text-to-video diffusion models to generate videos with desired motions.
- Release training and inference code.
- Release model weights.
- ...
(Row 1) Take two videos to train the proposed MotionDirector, respectively.
(Row 2) MotionDirector can generalize the learned motions to diverse appearances.
(Row 3) MotionDirector can mix the learned motion and appearance from different videos to generate new videos.
(Row 4) MotionDirector can animate a single image with learned motions.
Please refer to Project Page.
@article{zhao2023motiondirector,
title={MotionDirector: Motion Customization of Text-to-Video Diffusion Models},
author={Zhao, Rui and Gu, Yuchao and Wu, Jay Zhangjie and Zhang, David Junhao and Liu, Jiawei and Wu, Weijia and Keppo, Jussi and Shou, Mike Zheng},
journal={arXiv preprint arXiv:2310.08465},
year={2023}
}