Dinghow

llama3

The official Meta Llama 3 GitHub site

sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.

nvitop

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

HunyuanDiT

Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

api-for-open-llm

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

aria-ng-gui

一个 Aria2 图形界面客户端 | An Aria2 GUI for Windows & Linux & MacOS

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

ThunderKittens

Tile primitives for speedy kernels

Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

splatter-image

Official implementation of `Splatter Image: Ultra-Fast Single-View 3D Reconstruction' CVPR 2024

MInference

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

Bench2Drive

Closed-loop multi-ability evaluation of end-to-end autonomous driving algorithms

code-act

Official Repo for ICML 2024 paper "Executable Code Actions Elicit Better LLM Agents" by Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, Heng Ji.

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

ChunkLlama

[ICML'24] Data and code for our paper "Training-Free Long-Context Scaling of Large Language Models"

LLMDebugger

LDB: A Large Language Model Debugger via Verifying Runtime Execution Step by Step

SpeculativeDecodingPapers

📰 Must-read papers and blogs on Speculative Decoding ⚡️

DistServe

Disaggregated serving system for Large Language Models (LLMs).

ELM

[ECCV 2024] Embodied Understanding of Driving Scenarios

3dgcn

Convolution in the Cloud: Learning Deformable Kernels in 3D Graph Convolution Networks for Point Cloud Analysis

LM-Infinite

Implementation of paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"

PointMetaBase

This is a PyTorch implementation of PointMetaBase proposed by our paper "Meta Architecure for Point Cloud Analysis"

Point-SAM

Point-SAM: This is the official repository of "Point-SAM: Promptable 3D Segmentation Model for Point Clouds". We provide codes for running our demo and links to download checkpoints.

Grounded_3D-LLM

Code&Data for Grounded 3D-LLM with Referent Tokens

llm-scheduling-artifact

Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“

FLASHNN

DIF-Gaussian

MICCAI 2024: Learning 3D Gaussians for Extremely Sparse-View Cone-Beam CT Reconstruction

RecFlex

A recommendation model kernel optimizing system