iamlockelightning

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

PhotoMaker

PhotoMaker

EMO

Emote Portrait Alive: Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions

LWM

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

moondream

tiny vision language model

MiniCPM

MiniCPM-2B: An end-side LLM outperforming Llama2-13B.

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

LLaMA2-Accessory

An Open-source Toolkit for LLM Development

DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

coyo-dataset

COYO-700M: Large-scale Image-Text Pair Dataset

wit

WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.

MobileVLM

Strong and Open Vision Language Assistant for Mobile Devices

U-ViT

A PyTorch implementation of the paper "All are Worth Words: A ViT Backbone for Diffusion Models".

Bunny

A family of lightweight multimodal models.

VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 30+ benchmarks

ml-aim

This repository provides the code and model checkpoints of the research paper: Scalable Pre-training of Large Autoregressive Image Models

animate-anything

Fine-Grained Open Domain Image Animation with Motion Guidance

fast-DiT

Fast Diffusion Models with Transformers

LaVIT

LaVIT: Empower the Large Language Model to Understand and Generate Visual Content

maskgit

Official Jax Implementation of MaskGIT

DreamLLM

[ICLR 2024 Spotlight] DreamLLM: Synergistic Multimodal Comprehension and Creation

VisionLLaMA

VisionLLaMA: A Unified LLaMA Backbone for Vision Tasks

TaiSu

TaiSu（太素）--a large-scale Chinese multimodal dataset（亿级大规模中文视觉语言预训练数据集）

MMBench

Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

snip-dedup

M2PT

[CVPR'24] Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities

GVT

Official code for "What Makes for Good Visual Tokenizers for Large Language Models?".

wikiHow-VGSI

EMNLP 2021: Visual Goal-Step Inference using wikiHow

encode-once-and-decode-in-parallel