Yueeeeeeee

Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

fsdp_qlora

Training LLMs with QLoRA + FSDP

llm-rankers

Zero-shot Document Ranking with Large Language Models.

TokenReduction

Official PyTorch implementation of Which Tokens to Use? Investigating Token Reduction in Vision Transformers presented at ICCV 2023 NIVT workshop

depth-fm

DepthFM: Fast Monocular Depth Estimation with Flow Matching

graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system

ModSAR

[Under review] This is PyTorch-Lightning and Ray-Tune implementation of AE/AR reality check on self-attentive sequential recommenders

storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

loft

LOFT: A 1 Million+ Token Long-Context Benchmark

matmulfreellm

Implementation for MatMul-free LM.

ircot

Repository for Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions, ACL23

Ms-PoE

"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.

wit

WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.

xRAG

Source code for xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token

FlashRAG

⚡FlashRAG: A Python Toolkit for Efficient RAG Research

HippoRAG

HippoRAG is a novel RAG framework inspired by human long-term memory that enables LLMs to continuously integrate knowledge across external documents. RAG + Knowledge Graphs + Personalized PageRank.

RDRec

[ACL 2024] RDRec: Rationale Distillation for LLM-based Recommendation

MORE

LLMTest_NeedleInAHaystack

Doing simple retrieval from LLM models at various context lengths to measure accuracy

Mantis

Official code for Paper "Mantis: Multi-Image Instruction Tuning"

low-bit-optimizers

Low-bit optimizers for PyTorch

raptor

The official implementation of RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval

FILM

Official repo for "Make Your LLM Fully Utilize the Context"

Fed-MP

[NAACL 2024] Pytorch Implementation for "Open-Vocabulary Federated Learning with Multimodal Prototyping"

torchtitan

A native PyTorch Library for large model training

llama3

The official Meta Llama 3 GitHub site

sentence-transformers

Multilingual Sentence & Image Embeddings with BERT

Mamba4Rec

Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models

LLaMA-Factory

Efficiently Fine-Tune 100+ LLMs in WebUI (ACL 2024)

how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.