AndreJJXu

Vim

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

ShiArthur03

Diff-Foley

Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models

align_sd

Better Aligning Text-to-Image Models with Human Preference. ICCV 2023

chameleon

Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.

VGen

Official repo for VGen: a holistic video generation ecosystem for video generation building on diffusion models

RevIN

RevIN: Reversible Instance Normalization For Accurate Time-series Forecasting Against Distribution Shift

ai-audio-datasets

AI Audio Datasets (AI-ADS) 🎵, including Speech, Music, and Sound Effects, which can provide training data for Generative AI, AIGC, AI model training, intelligent audio tool development, and audio applications.

audio2photoreal

Code and dataset for photorealistic Codec Avatars driven from audio

T2I-CompBench

[Neurips 2023] T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation

pykan

Kolmogorov Arnold Networks

cobra

Cobra: Extending Mamba to Multi-modal Large Language Model for Efficient Inference

LSLD

DG-SCT

NeurIPS'2023 official implementation code

LAVISH

Vision Transformers are Parameter-Efficient Audio-Visual Learners

audio-dataset

Audio Dataset for training CLAP and other models

CLAP

Contrastive Language-Audio Pretraining

RPG-DiffusionMaster

[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (PRG)

Panda-70M

[CVPR 2024] Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

ViTamin

[CVPR 2024] Official implementation of "ViTamin: Designing Scalable Vision Models in the Vision-language Era"

SceneWiz3D

[CVPR 2024] SceneWiz3D: Towards Text-guided 3D Scene Composition

MGM

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

ControlNet-v1-1-nightly

Nightly release of ControlNet 1.1

ControlNet

Let us control diffusion models!

dreambooth

blended-latent-diffusion

Official implementation for "Blended Latent Diffusion" [SIGGRAPH 2023]

SyncDiffusion

Official implementation of SyncDiffusion.

MultiDiffusion

Official Pytorch Implementation for "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" presenting "MultiDiffusion" (ICML 2023)

ModalBiasAVSR

Offical implementation of the CVPR 2024 paper: A Study of Dropout-Induced Modality Bias on Robustness to Missing Video.

clotho-dataset

Python code for handling the Clotho dataset.