Danield21's repositories
academicpages.github.io
Github Pages template for academic personal websites, forked from mmistakes/minimal-mistakes
Adan
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
AISystem
AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术
AQLM
Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf
Awesome-LLM-Compression
Awesome LLM compression research papers and tools.
Awesome-LLM-Inference
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
Awesome-LLM-Survey
An Awesome Collection for LLM Survey
awsome-distributed-training
Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.
DynMoE
[Preprint] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
Jetson-NANO-4GB
Jetson NANO 4GB B01/SUB Developer Kit
Lamini-Memory-Tuning
Banishing LLM Hallucinations Requires Rethinking Generalization
MC-SMoE
[ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
nanotron
Minimalistic large language model 3D-parallelism training
redco
NAACL '24 (Demo) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
SECURE
ACL'2024-Main: Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Models
strictjson
A Strict JSON Framework for LLM Outputs
StructuredFFN
The official code of "Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers"
Subspace-Tuning
A generalized framework for subspace tuning methods in parameter efficient fine-tuning.
Teleportation-Optimization
[ICLR 2024] Improving Convergence and Generalization Using Parameter Symmetries
unsloth
Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory