xueyuuu

mllm-npu

mllm-npu: training multimodal large language models on Ascend NPUs

line_profiler

Line-by-line profiling for Python

LWM

mamba

Mamba SSM architecture

Yi

A series of large language models trained from scratch by developers @01-ai

PoSE

Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)

COSMA

Distributed Communication-Optimal Matrix-Matrix Multiplication Algorithm

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

pandora

潘多拉，一个让你呼吸顺畅的ChatGPT。Pandora, a ChatGPT that helps you breathe smoothly.

mindpet

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

oneflow

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

llama

Inference code for Llama models

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

flash-attention

Fast and memory-efficient exact attention

fastmoe

A fast MoE impl for PyTorch

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

libai

LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training

ColossalAI

Making large AI models cheaper, faster and more accessible

mindspore

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

light-dist-gnn

AI-System

System for AI Education Resource.

CodeClass

在线阅读更方便：https://coderlemon.com/ 计算机编程学习路线 + 学习资源，最全面的编程学习课堂，从小白到架构师，关于编程所有你需要掌握的内容都在这里，持续更新中！

PyTorch-ParameterServer

An implementation of parameter server framework in PyTorch RPC.

PyG-OGB-Tricks

Bags of Tricks in OGB (node classification) with GCNs.

PipeGCN

[ICLR 2022] "PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication" by Cheng Wan, Youjie Li, Cameron R. Wolfe, Anastasios Kyrillidis, Nam Sung Kim, Yingyan Lin

google-research

Google Research

torch-quiver

PyTorch Library for Low-Latency, High-Throughput Graph Learning on GPUs.

DrakeTyporaTheme

十二种主题风格 - Material Google JetBrains Vue Juejin Purple Ayu Dark

onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator