Marko Kabić's repositories
alpa
Auto parallelization for large-scale neural networks
apex
A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
attention-is-all-you-need-pytorch
A PyTorch implementation of the Transformer model in "Attention is All You Need".
ColossalAI
Colossal-AI: A Unified Deep Learning System for Big Model Era
COSTA
Distributed Communication-Optimal Shuffle and Transpose Algorithm
cudf
cuDF - GPU DataFrame Library
cylon
Cylon is a fast, scalable, distributed memory, parallel runtime with a Pandas like DataFrame.
FasterTransformer
Transformer related optimization, including BERT, GPT
FBGEMM
FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/
flash-attention
Fast and memory-efficient exact attention
flax
Flax is a neural network library for JAX that is designed for flexibility.
gavel
Code for "Heterogenity-Aware Cluster Scheduling Policies for Deep Learning Workloads", which appeared at OSDI 2020
google-research
Google Research
marius
Large scale embeddings on a single machine.
mesh
Mesh TensorFlow: Model Parallelism Made Easier
mesh-transformer-jax
Model parallel transformers in JAX and Haiku
minGPT
A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
parallelformers
Parallelformers: An Efficient Model Parallelization Toolkit for Deployment
pytorch3d
PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
query-engine
LingoDB: A new analytical database system that blurs the lines between databases and compilers.
semiprof
Simple thread safe annotation based C++ profiler.
snn_toolbox
Toolbox for converting analog to spiking neural networks (ANN to SNN), and running them in a spiking neuron simulator.
spack
A flexible package manager that supports multiple versions, configurations, platforms, and compilers.
sql-parser
SQL Parser for C++. Building C++ object structure from SQL statements.
transformer-from-scratch
Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
trax
Trax — Deep Learning with Clear Code and Speed