wefwefWEF2

DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

deep-seek

LLM powered retrieval engine designed to process a ton of sources to collect a comprehensive list of entities.

MiniGPT-4

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

imp

a family of highly capabale yet efficient large multimodal models

Bunny

A family of lightweight multimodal models.

FuseAI

FuseLLM & FuseChat Project

H-SAM

MemSAM

[CVPR 2024 Oral] MemSAM: Taming Segment Anything Model for Echocardiography Video Segmentation.

mllm

Fast Multimodal LLM on Mobile Devices

Awesome-LLM-System-Papers

LLMSpeculativeSampling

Fast inference from large lauguage models via speculative decoding

SwiftSage

SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks

LLaVA-UHD

LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images

MiniCPM-V

MiniCPM-Llama3-V 2.5: A GPT-4V Level Multimodal LLM on Your Phone

MADTP

MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer

VTW

PSALM

This is an official implementation for "PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model"

JiuTian

[CVPR 2024] JiuTian, a Multimodal Large Language Model from HITSZ

LISA

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

crg

PyTorch code for "Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training"

shikra

annotated_deep_learning_paper_implementations

🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

ml-ferret

Agent-FLAN

[ACL2024 Findings] Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models

MiniSora-DiT

minisora-DiT, a DiT reproduction based on XTuner from the open source community MiniSora

OpenAI-CLIP-Feature

An easy to use, user-friendly and efficient code for extracting OpenAI CLIP (Global/Grid) features from image and text respectively.

MaskFormer

Per-Pixel Classification is Not All You Need for Semantic Segmentation (NeurIPS 2021, spotlight)

LAMM

[NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI Agents

Osprey

[CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"

unify-parameter-efficient-tuning

Implementation of paper "Towards a Unified View of Parameter-Efficient Transfer Learning" (ICLR 2022)