SparkJiao

LLaMA-Factory

Unify Efficient Fine-Tuning of 100+ LLMs

DeepSeek-Coder

DeepSeek Coder: Let the Code Write Itself

trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Pearl

A Production-ready Reinforcement Learning AI Agent Library brought by the Applied Reinforcement Learning team at Meta.

EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

LLMAgentPapers

Must-read Papers on LLM Agents.

megablocks

open-instruct

dlrover

DLRover: An Automatic Distributed Deep Learning System

DeepSeek-MoE

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

megablocks-public

llama-moe

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

alpaca_farm

A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

Awesome-LLMs-Evaluation-Papers

The papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey.

EAGLE

Official Implementation of EAGLE

MS-AMP

Microsoft Automatic Mixed Precision Library

lumos

Code and data for "Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs"

CoLLiE

Collaborative Training of Large Language Models in an Efficient Way

Awesome-Reasoning-Foundation-Models

✨✨Latest Papers and Benchmarks in Reasoning with Foundation Models

llamafia.github

Q-Instruct

②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

OpenSource-LLMs-better-than-OpenAI

Listing all reported open-source LLMs achieving a higher score than proprietary, paying OpenAI models (ChatGPT, GPT-4).

RLCD

Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment

ShareGPTs

C-VQA

Counterfactual Reasoning VQA Dataset

gpt_paper_assistant

GPT4 based personalized ArXiv paper assistant bot