demonatic

long-context-attention

Sequence Parallel Attention for Long Context LLM Model Training and Inference

veScale

A PyTorch Native LLM Training Framework

NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

CUDATracePreload

CUDATracePreload is a dynamic tracing tool for CUDA and NCCL API calls.

Megatron-LM

Ongoing research training transformer models at scale

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

triton

Development repository for the Triton language and compiler

einops

Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)

MagnumIO

Magnum IO community repo

llama.cpp

LLM inference in C/C++

LWM

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

LLMs_interview_notes

该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题

TensorNVMe

A Python library transfers PyTorch tensors between CPU and NVMe

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

learn-nlp-with-transformers

we want to create a repo to illustrate usage of transformers in chinese

obsidian-better-export-pdf

Obsidian PDF export enhancement plugin

flash-attention

Fast and memory-efficient exact attention

PatrickStar

PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP and democratizes AI for everyone.

AISystem

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

ColossalAI

Making large AI models cheaper, faster and more accessible

stdgpu

stdgpu: Efficient STL-like Data Structures on the GPU

obsidian-quickshare

📝 An Obsidian plugin for sharing encrypted Markdown notes on the web. Zero configuration required.

libalgebra

Fast C header-only library for popcnt, pospopcnt, and set algebraic operations

cuCollections