EPFL Machine Learning and Optimization Laboratory

sent2vec

General purpose unsupervised sentence representations

ML_course

EPFL Machine Learning Course, Fall 2023

attention-cnn

Source code for "On the Relationship between Self-Attention and Convolutional Layers"

OptML_course

EPFL Course - Optimization for Machine Learning - CS-439

landmark-attention

Landmark Attention: Random-Access Infinite Context Length for Transformers

federated-learning-public-code

collaborative-attention

Code for Multi-Head Attention: Collaborate Instead of Concatenate

powersgd

Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727

disco

Decentralized & federated privacy-preserving ML training, using p2p networking, in JS

dynamic-sparse-flash-attention

DenseFormer

llm-baselines

optML-pku

summer school materials

error-feedback-SGD

SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847

pam

REQ

relaysgd

Code for the paper “RelaySum for Decentralized Deep Learning on Heterogeneous Data”

topology-in-decentralized-learning

Code related to ’Beyond spectral gap: The role of the topology in decentralized learning‘.

quasi-global-momentum

getting-started

byzantine-robust-decentralized-optimizer

easy-summary

difficulty-guided text summarization

text_to_image_generation

ghost-noise

personalized-collaborative-llms

phantomedicus

MedSurge: medical survey generator

cifar

MLO internal cifar 10 / 100 default implementation / reference implementation. single machine, variable batch sizes, allowing maybe gradient compression. need to have clear documentation to make it easy to use, and so that we don't loose time with looking for hyperparameters. we can later keep it in sync with mlbench too, but self-contained is even better

DoGE

Codebase for ICML submission "DOGE: Domain Reweighting with Generalization Estimation"

epfml-utils

Tools for experimentation and using run:ai. The aim is for these to be small self-contained utilities that are used by multiple people.

getting-started-lauzhack