zyqCSL

AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

langchain

🦜🔗 Build context-aware reasoning applications

llama.cpp

LLM inference in C/C++

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

stanford_alpaca

Code and documentation to train Stanford's Alpaca models, and generate the data.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

llama3

The official Meta Llama 3 GitHub site

MemGPT

Letta (fka MemGPT) is a framework for creating stateful LLM services.

StreamDiffusion

StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

llama-cpp-python

Python bindings for llama.cpp

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

lm-evaluation-harness

A framework for few-shot evaluation of language models.

FasterTransformer

Transformer related optimization, including BERT, GPT

Awesome-Incremental-Learning

Awesome Incremental Learning

fairscale

PyTorch extensions for high performance and large scale training.

Ax

Adaptive Experimentation Platform

flashinfer

FlashInfer: Kernel Library for LLM Serving

torchgpipe

A GPipe implementation in PyTorch

rouge

A full Python Implementation of the ROUGE Metric (not a wrapper)

Freeflow

High performance container overlay networks on Linux. Enabling RDMA (on both InfiniBand and RoCE) and accelerating TCP to bare metal performance. Freeflow requires zero modification on application code/binary.

aws-neuron-sdk

Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and integrated with your favorite AWS services

Awesome-LLM-Eval

Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表，主要面向基础大模型评测，旨在探求生成式AI的技术边界.

pipedream

k8s-dra-driver

Dynamic Resource Allocation (DRA) for NVIDIA GPUs in Kubernetes

ML-Murphy

Complete solutions for exercises and MATLAB example codes for "Machine Learning: A Probabilistic Perspective" 1/e by K. Murphy