arthur-qiu / FreeTraj

Code for FreeTraj, a tuning-free method for trajectory-controllable video generation

FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models

           

Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia*, and Ziwei Liu*
(* corresponding author)

🔆 Introduction

🤗🤗🤗 FreeTraj is a tuning-free method for trajectory-controllable video generation based on pre-trained video diffusion models.

Showcases (320x512)

"A chihuahua in an astronaut suit floating in the universe, cinematic lighting, glow effect."	"A swan floating gracefully on a lake."	"A corgi running on the grassland on the grassland."

"A barrel floating in a river."	"A dog running across the garden, photorealistic, 4k."	"A helicopter hovering above a cityscape."

📝 Changelog

• [2024.06.25]: 🔥🔥 Code coming soon (under cleaning)

🧰 Models

Model	Resolution	Checkpoint	Description
VideoCrafter2 (Text2Video)	320x512	Hugging Face

😉 Citation

@misc{qiu2024freetraj,
      title={FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models}, 
      author={Haonan Qiu and Zhaoxi Chen and Zhouxia Wang and Yingqing He and Menghan Xia and Ziwei Liu},
      year={2024},
      eprint={2406.16863},
      archivePrefix={arXiv}
}