TKONIY

vmcache

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

MSVBASE

MSVBASE is a system that efficiently supports complex queries of both approximate similarity search and relational operators. It integrates high-dimensional vector indices into PostgreSQL, a relational database to facilitate complex approximate similarity queries.

GPUDB-Prefetch

Source code of our DaMoN@SIGMOD 2024 paper "How Does Software Prefetching Work on GPU Query Processing?"

Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

NExT-QA

NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (CVPR'21)

TriForce

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

FastLanesGPU

Accelerating GPU Data Processing using FastLanes Compression

dash

Scalable Hashing on Persistent Memory

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

flashinfer

FlashInfer: Kernel Library for LLM Serving

Yi

A series of large language models trained from scratch by developers @01-ai

punica

Serving multiple LoRA finetuned LLM as one

Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

megalodon

Reference implementation of Megalodon 7B model

github-markdown-toc

Easy TOC creation for GitHub README.md

sarathi-serve

A low-latency & high-throughput serving engine for LLMs

ringattention

Transformers with Arbitrarily Large Context

SwiftTransformer

High performance Transformer implementation in C++.

DistServe

Disaggregated serving system for Large Language Models (LLMs).

DALLE2-pytorch

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

Awesome-LLM-Long-Context-Modeling

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

LLMAgentPapers

Must-read Papers on LLM Agents.

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.