jsw-zorro

CitationMap

A simple pip-installable Python tool to generate your own HTML citation world map from your Google Scholar ID.

sglang

SGLang is yet another fast serving framework for large language models and vision language models.

ServerlessLLM

Cost-efficient and fast multi-LLM serving.

how-to-learn-deep-learning-framework

how to learn PyTorch and OneFlow

pytorch-cppcuda-tutorial

tutorial for writing custom pytorch cpp+cuda kernel, applied on volume rendering (NeRF)

lectures

Material for cuda-mode lectures

vidur

A large-scale simulation framework for LLM inference

sarathi-serve

A low-latency & high-throughput serving engine for LLMs

calculate-flops.pytorch

The calflops is designed to calculate FLOPs、MACs and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model)

DistServe

Disaggregated serving system for Large Language Models (LLMs).

torchtitan

A native PyTorch Library for large model training

long-context-attention

Sequence Parallel Attention for Long Context LLM Model Training and Inference

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

llama3-from-scratch

llama3 implementation one matrix multiplication at a time

LLMServingPerfEvaluator

guidance

A guidance language for controlling large language models.

S-LoRA

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

prometheus-eval

Evaluate your LLM's response with Prometheus and GPT4 💯

attorch

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

llama3

The official Meta Llama 3 GitHub site

torchtune

A Native-PyTorch Library for LLM Fine-tuning

simple-evals

VAR

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

llm.c

LLM training in simple, raw C/CUDA

Primo

Primo: Practical Learning-Augmented Systems with Interpretable Models

outlines

Structured Text Generation

SWE-agent

SWE-agent takes a GitHub issue and tries to automatically fix it, using GPT-4, or your LM of choice. It solves 12.47% of bugs in the SWE-bench evaluation set and takes just 1 minute to run.

scattermoe

Triton-based implementation of Sparse Mixture of Experts.

LLM-Blender

[ACL2023] We introduce LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the diverse strengths of multiple open-source LLMs. LLM-Blender cut the weaknesses through ranking and integrate the strengths through fusing generation to enhance the capability of LLMs.

Awesome-Efficient-LLM

A curated list for Efficient Large Language Models