MengShen0709

openchat

OpenChat: Advancing Open-source Language Models with Imperfect Data

Multi-Modality-Arena

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!

ft-pali-gemma

Notebooks for fine tuning pali gemma

MiniCPM-V

MiniCPM-Llama3-V 2.5: A GPT-4V Level Multimodal LLM on Your Phone

data_management_LLM

Collection of training data management explorations for large language models

fast-detect-gpt

Code base for "Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature".

ChatTTS

A generative speech model for daily dialogue.

Self-Filter

HallE_Control

HallE-Control: Controlling Object Hallucination in LMMs

OPERA

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

MiniGPT-4

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

dlrm

An implementation of a deep learning recommendation model (DLRM)

Multimodal-AND-Large-Language-Models

Paper list about multimodal and large language models, only used to record papers I read in the daily arxiv for personal needs.

open-instruct

LESS

[ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning

prismatic-vlms

A flexible and efficient codebase for training visually-conditioned language models (VLMs)

vlm-evaluation

VLM Evaluation: Benchmark for VLMs, spanning text generation tasks from VQA to Captioning

Dromedary

Dromedary: towards helpful, ethical and reliable LLMs.

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

InternLM-XComposer

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

DataOptim

A collection of visual instruction tuning datasets.

Bunny

A family of lightweight multimodal models.

bmmal

[ACMMM 2023] BMMAL: Towards Balanced Active Learning for Multimodal Classification

faiss

A library for efficient similarity search and clustering of dense vectors.

ijepa

Official codebase for I-JEPA, the Image-based Joint-Embedding Predictive Architecture. First outlined in the CVPR paper, "Self-supervised learning from images with a joint-embedding predictive architecture."

open_clip

An open source implementation of CLIP.

TrustLLM

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

GPT-SoVITS

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

TalkLip

sas-data-efficient-contrastive-learning

Official repository for SAS Data Efficient Contrastive Learning ICML '23