Gary-code

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

llm-action

本项目旨在分享大模型相关技术原理以及实战经验。

MiniCPM

MiniCPM-2B: An end-side LLM outperforming Llama2-13B.

Qwen-VL

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

dive-into-llms

《动手学大模型Dive into LLMs》系列编程实践教程

PixArt-alpha

PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

MoE-LLaVA

Mixture-of-Experts for Large Vision-Language Models

Qwen-Audio

The official repo of Qwen-Audio (通义千问-Audio) chat & pretrained large audio language model proposed by Alibaba Cloud.

OMG-Seg

OMG-LLaVA and OMG-Seg codebase

VideoMamba

[ECCV2024] VideoMamba: State Space Model for Efficient Video Understanding

groundingLMM

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

LaVIN

[NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"

KG-MM-Survey

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

OPERA

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

RLHF-V

[CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

LLM-generated-Text-Detection

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions.

QuRating

[ICML 2024] Selecting High-Quality Data for Training Language Models

InCTRL

Official implementation of CVPR'24 paper 'Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts'.

LearnDeepSpeed

DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）

Pink

Pink: Unveiling the Power of Referential Comprehension for Multi-modal LLMs

text-to-cad-ui

A lightweight UI for interfacing with the Zoo text-to-cad API, built with SvelteKit.

AnyDoor

AnyDoor: Test-Time Backdoor Attacks on Multimodal Large Language Models

LM-Science-Tutor

C-VQA

Counterfactual Reasoning VQA Dataset

Sniffer

SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection

CUTI-Domain

UNTL

EMNLP'2022: Unsupervised Non-transferable Text Classification

Easy_LLM_Tool

简单的大模型微调工具包

D-VQG

[TCSVT 2024] The released code of paper "Video Question Generation for Dynamic Changes"