chiennv2000

minichatgpt

minichatgpt - To Train ChatGPT In 5 Minutes

Qwen

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

fmengine

Utilities for Training Very Large Models

system-design-101

Explain complex systems using visuals and simple terms. Help you prepare for system design interviews.

unidiffuser

Code and models for the paper "One Transformer Fits All Distributions in Multi-Modal Diffusion"

automatic_prompt_engineer

mistral-inference

Official inference library for Mistral models

InternLM

Official release of InternLM2 7B and 20B base and chat models. 200K context support

llm-foundry

LLM training code for Databricks foundation models

pyllama

LLaMA: Open and Efficient Foundation Language Models

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

LLMsPracticalGuide

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

MINDER

PromptKG

PromptKG Family: a Gallery of Prompt Learning & KG-related research works, toolkits, and paper-list.

EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

t5x

litgpt

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

yarn

YaRN: Efficient Context Window Extension of Large Language Models

llama_index

LlamaIndex is a data framework for your LLM applications

RRHF

[NIPS2023] RRHF & Wombat

FlagEmbedding

Retrieval and Retrieval-augmented LLMs

awesome-django

The Best Django Resource, Awesome Django for mature packages.

DeepSpeedExamples

Example models using DeepSpeed

sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric

Okapi

Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback

mlmm-evaluation

Multilingual Large Language Models Evaluation Benchmark

annotated_deep_learning_paper_implementations

🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

labml

🔎 Monitor deep learning model training and hardware usage from your mobile phone 📱

RoFormer_pytorch

RoFormer V1 & V2 pytorch

attention_with_linear_biases

Code for the ALiBi method for transformer language models (ICLR 2022)