VincentDENGP

Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

Video-Bench

A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models!

MLVU

🔥🔥MLVU: Multi-task Long Video Understanding Benchmark

LongVA

Long Context Transfer from Language to Vision

open_clip

An open source implementation of CLIP.

Kolors

Kolors Team

HallusionBench

[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

MathVista

MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts

MM-Vet

MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities (ICML 2024)

InternLM-XComposer

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

GLM-4

GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型

Video-Streaming-Research-Papers

Research materials about multimedia network and system, including paper list, tools, etc.

llama

Inference code for Llama models

tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Step-DPO

Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"

cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

LLM101n

LLM101n: Let's build a Storyteller

fms-fsdp

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.

Video-MME

✨✨Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

litdata

Transform datasets at scale. Optimize datasets for fast AI model training.

OpenRLHF

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & Mixtral)

MINT-1T

MINT-1T: A one trillion token multimodal interleaved dataset.

DynamiCrafter

[ECCV 2024] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors

minisora

MiniSora: A community aims to explore the implementation path and future development direction of Sora.

OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

bolei_awesome_posters

CVPR and NeurIPS poster examples and templates. May we have in-person poster session soon!

LlamaGen

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

IG-VLM

MixEval

The official evaluation suite and dynamic data release for MixEval.

Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.