wofmanaf

Volcano

[NAACL 2024] Vision language model that reduces hallucinations through self-feedback guided revision. Visualizes attentions on image features for interpretation.

Dataset-Pruning

Dataset pruning for ImageNet and LAION-2B.

Pix2Text

An Open-Source Python3 tool for recognizing layouts, tables, math formulas (LaTeX), and text in images, converting them into Markdown format. A free alternative to Mathpix, empowering seamless conversion of visual content into text-based representations. 80+ languages are supported.

jepa

PyTorch code and models for V-JEPA self-supervised learning from video.

Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

OLMo

Modeling, training, eval, and inference code for OLMo

Kerlinn.github.io

VILA

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

TimeChat

[CVPR 2024] TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

GaLore

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

bigcode-dataset

SPIN

The official implementation of Self-Play Fine-Tuning (SPIN)

EfficientSAM

EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything

MobileAgent

Mobile-Agent: The Powerful Mobile Device Operation Assistant Family

UltraEval

An open source framework for evaluating foundation models.

MiniCPM

MiniCPM-2B: An end-side LLM outperforming Llama2-13B.

Monkey

【CVPR 2024 Highlight】Monkey (LMM): Image Resolution and Text Label Are Important Things for Large Multi-modal Models

Data-for-LaTeX_OCR

LaTeX OCR 的数据仓库

TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

EmbodiedScan

[CVPR 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

donut

Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022

alignment-handbook

Robust recipes to align language models with human and AI preferences

direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

datasketch

MinHash, LSH, LSH Forest, Weighted MinHash, HyperLogLog, HyperLogLog++, LSH Ensemble and HNSW

UReader

MNBVC

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

open_clip

An open source implementation of CLIP.

llama.onnx

LLaMa/RWKV onnx models, quantization and testcase