Dinghow

aria-ng-gui

一个 Aria2 图形界面客户端 | An Aria2 GUI for Windows & Linux & MacOS

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

FLASHNN

api-for-open-llm

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口

MInference

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

Point-SAM

Point-SAM: This is the official repository of "Point-SAM: Promptable 3D Segmentation Model for Point Clouds". We provide codes for running our demo and links to download checkpoints.

Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

DIF-Gaussian

MICCAI 2024: Learning 3D Gaussians for Extremely Sparse-View Cone-Beam CT Reconstruction

RecFlex

A recommendation model kernel optimizing system

DistServe

Disaggregated serving system for Large Language Models (LLMs).

Bench2Drive

Closed-loop multi-ability evaluation of end-to-end autonomous driving algorithms

LLMDebugger

LDB: A Large Language Model Debugger via Verifying Runtime Execution Step by Step

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

3dgcn

Convolution in the Cloud: Learning Deformable Kernels in 3D Graph Convolution Networks for Point Cloud Analysis

sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.

nvitop

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

ELM

[ECCV 2024] Embodied Understanding of Driving Scenarios

Grounded_3D-LLM

Code&Data for Grounded 3D-LLM with Referent Tokens

SpeculativeDecodingPapers

📰 Must-read papers and blogs on Speculative Decoding ⚡️

HunyuanDiT

Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

ThunderKittens

Tile primitives for speedy kernels

DeepSeek-V2

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

splatter-image

Official implementation of `Splatter Image: Ultra-Fast Single-View 3D Reconstruction' CVPR 2024

code-act

Official Repo for ICML 2024 paper "Executable Code Actions Elicit Better LLM Agents" by Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, Heng Ji.

LM-Infinite

Implementation of paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"

llm-scheduling-artifact

Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“

PointMetaBase

This is a PyTorch implementation of PointMetaBase proposed by our paper "Meta Architecure for Point Cloud Analysis"

ChunkLlama

[ICML'24] Data and code for our paper "Training-Free Long-Context Scaling of Large Language Models"

llama3

The official Meta Llama 3 GitHub site