jpWang

ICL-D3IE

Code for ICCV 2023 Paper : “ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction”

UReader

Monkey

【CVPR 2024 Highlight】Monkey (LMM): Image Resolution and Text Label Are Important Things for Large Multi-modal Models

LLM-in-Vision

Recent LLM-based CV and related works. Welcome to comment/contribute!

Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

groundingLMM

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

NExT-GPT

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

open_clip

An open source implementation of CLIP.

SEED

Official implementation of SEED-LLaMA (ICLR 2024).

PickScore

improved-aesthetic-predictor

CLIP+MLP Aesthetic Score Predictor

LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".

stable-diffusion-webui

Stable Diffusion web UI

trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

nltk_data

NLTK Data

nltk

NLTK Source

Union14M

[ICCV 2023] Code base for Revisiting Scene Text Recognition: A Data Perspective

diffusiondb

A large-scale text-to-image prompt gallery dataset based on Stable Diffusion

instruct-pix2pix

MagicBrush

[NeurIPS'23] "MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing".

gill

🐟 Code and models for the NeurIPS 2023 paper "Generating Images with Multimodal Language Models".

qlora

QLoRA: Efficient Finetuning of Quantized LLMs

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

recognize-anything

Open-source and strong foundation image recognition models.

M6Doc

ImageBind

ImageBind One Embedding Space to Bind Them All

mPLUG-Owl

mPLUG-Owl & mPLUG-Owl2: Modularized Multimodal Large Language Model

FindTheChatGPTer

ChatGPT爆火，开启了通往AGI的关键一步，本项目旨在汇总那些ChatGPT的开源平替们，包括文本大模型、多模态大模型等，为大家提供一些便利

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

MiniGPT-4

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)