litwellchi

3D-VLA

[ICML 2024] 3D-VLA: A 3D Vision-Language-Action Generative World Model

phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

Awesome-Video-Robotic-Papers

This repository compiles a list of papers related to the application of video technology in the field of robotics! Star⭐ the repo and follow me if you like what you see🤩.

videocrafter-training-pytorch

Training code for the videocrafter.

MMTrail-Pytorch

[Arxiv 2024] Official code for MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions

awesome-diffusion-model-in-rl

A curated list of Diffusion Model in RL resources (continually updated)

1xgpt

world modeling challenge for humanoid robots

Pointcept

Pointcept: a codebase for point cloud perception research. Latest works: PTv3 (CVPR'24 Oral), PPT (CVPR'24), OA-CNNs (CVPR'24), MSC (CVPR'23)

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

Awesome-Embodied-AI

A curated list of awesome papers on Embodied AI and related research/industry-driven resources.

AlignProp

AlignProp uses direct reward backpropogation for the alignment of large-scale text-to-image diffusion models. Our method is 25x more sample and compute efficient than reinforcement learning methods (PPO) for finetuning Stable Diffusion

MMTrail

[Arxiv 2024] Official code for MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions

1d-tokenizer

This repo contains the code for our paper An Image is Worth 32 Tokens for Reconstruction and Generation

MMWorld

Official repo of the paper "MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos"

calvin

CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

open_flamingo

An open-source framework for training large multimodal models.

Pandora

Pandora: Towards General World Model with Natural Language Actions and Video States

Era3D

Awesome-Video-Datasets

Video datasets

lvm_datapipe

data pipeline code of large video generation model

Seeing-and-Hearing

[CVPR 2024] Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners

SVD_Xtend

Stable Video Diffusion Training Code and Extensions.

Latte

Latte: Latent Diffusion Transformer for Video Generation.

LaVIT

LaVIT: Empower the Large Language Model to Understand and Generate Visual Content

MMDialog

The official site of paper MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation

magvit

Official JAX implementation of MAGVIT: Masked Generative Video Transformer

magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

AnimateDiff

Official implementation of AnimateDiff.

M2Chat

Awesome-LLMs-meet-Multimodal-Generation

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).