liuhatry's repositories
Aluminum
Aluminum GPU-aware Communication Library
Language:C++NOASSERTION000
Language:C++000
fastmoe
A fast MoE impl for PyTorch
Language:PythonApache-2.0000
liuhatry.github.io
liuhatry's Blog
Language:HTML000
Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Language:PythonNOASSERTION000
Language:C++000
Language:C++000
tutel
Tutel MoE: An Optimized Mixture-of-Experts Implementation
Language:PythonMIT000
tutorials
Bagua tutorials.
Language:ShellMIT000
xflow
Distributed LR、 FM model on Parameter Server. FTRL and SGD Optimization Algorithm.
Language:C++000