tuofeilunhifi

ollama

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

pykan

Kolmogorov Arnold Networks

MiniCPM-V

MiniCPM-Llama3-V 2.5: A GPT-4V Level Multimodal LLM on Your Phone

llama-cpp-python

Python bindings for llama.cpp

IC-Light

More relighting!

efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

SupContrast

PyTorch implementation of "Supervised Contrastive Learning" (and SimCLR incidentally)

ColBERT

ColBERT: state-of-the-art neural search (SIGIR'20, TACL'21, NeurIPS'21, NAACL'22, CIKM'22, ACL'23, EMNLP'23)

PyContrast

PyTorch implementation of Contrastive Learning methods

DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

CogVLM2

GPT4V-level open-source multi-modal model based on Llama3-8B

mPLUG-DocOwl

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

VILA

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

pytorch-randaugment

Unofficial PyTorch Reimplementation of RandAugment.

VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support GPT-4v, Gemini, QwenVLPlus, 50+ HF models, 20+ benchmarks

Grounding-DINO-1.5-API

API for Grounding DINO 1.5: IDEA Research's Most Capable Open-World Object Detection Model Series

Groma

Grounded Multimodal Large Language Model with Localized Visual Tokenization

TinyLLaVA_Factory

A Framework of Small-scale Large Multimodal Models

MultimodalOCR

On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)

prismatic-vlms

A flexible and efficient codebase for training visually-conditioned language models (VLMs)

scaling_on_scales

When do we not need larger vision models?

ScreenAI

Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"

OmniFusion

OmniFusion — a multimodal model to communicate using text and images

Retrieval-Augmented-Visual-Question-Answering

This is the official repository for Retrieval Augmented Visual Question Answering

MMBench

Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

DeLVM

MoVA

MoVA: Adapting Mixture of Vision Experts to Multimodal Context

Awesome-Vision-Mamba

✨✨Latest Papers on Vision Mamba and Related Areas

fld

PyTorch code for FLD (Feature Likelihood Divergence), FID, KID, Precision, Recall, etc. using DINOv2, InceptionV3, CLIP, etc.

CLIP-KD

[CVPR-2024] Official implementations of CLIP-KD: An Empirical Study of CLIP Model Distillation