Orion-Zheng

Vript

alignment-handbook

Robust recipes to align language models with human and AI preferences

Skywork-MoE

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

llamafia.github

MixEval

The official evaluation suite and dynamic data release for MixEval.

LLMPruner

ChatTTS

A generative speech model for daily dialogue.

pytorch-learning

learning notes when learning the source code of pytorch

multi_gpu_training

web

React web interface for the OpenDota platform

dota2py

Python tools for Dota 2

compendium

Dota 2 replay knowledge in book form.

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

rho

Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.

dota2-clarity

Custom console scripts for Dota 2.

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

DeepLearningSystem

AI Infra主要是指AI的基础建设，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术。

JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

llm-colosseum

Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM

llm-compressive

Longitudinal Evaluation of LLMs via Data Compression

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

dbrx

Code examples and resources for DBRX, a large language model developed by Databricks

Neural-Network-Parameter-Diffusion

We introduce a novel approach for parameter generation, named neural network parameter diffusion (p-diff), which employs a standard latent diffusion model to synthesize a new set of parameters

G_VBSM_Dataset_Condensation

[CVPR2024 highlight] Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching (G-VBSM)

minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

dora

Implementation of DoRA

zero_nlp

中文nlp解决方案(大模型、数据、模型、训练、推理)

onboarding

Onboarding guide to Jimmy Lin's research group at the University of Waterloo