ShenDezhou

llm-playground

Experiments with open source LLMs

litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

ollama

Get up and running with Llama 3, Mistral, Gemma, and other large language models.

LoftQ

LDDL

Distributed preprocessing and data loading for language datasets

character-bert-pretraining

Code for pre-training CharacterBERT models (as well as BERT models).

ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI

LLM-Workshop

LLM Workshop by Sourab Mangrulkar

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

ColossalAI

Making large AI models cheaper, faster and more accessible

Llama2-chinese

Llama2 chinese finetuning

llama2-lora-fine-tuning

llama2 finetuning with deepspeed and lora

llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

MemGPT

Create LLM agents with long-term memory and custom tools 📚🦙

localpilot

TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

vscode-extension-samples

Sample code illustrating the VS Code extension API.

MetaGPT

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

llama.cpp

LLM inference in C/C++

zero_shot_cot

Prod Env

natural-instructions

Expanding natural instructions

nanoT5

Fast & Simple repository for pre-training and fine-tuning T5-style models

bert_distill

BERT distillation（基于BERT的蒸馏实验 ）

beir

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

duckdb-pgq

DuckDB is an in-process SQL OLAP Database Management System

tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

duckdb

DuckDB is an in-process SQL OLAP Database Management System

arrow-tools

A collection of handy CLI tools to convert CSV and JSON to Apache Arrow and Parquet