EPFL Machine Learning and Optimization Laboratory's repositories
OptML_course
EPFL Course - Optimization for Machine Learning - CS-439
attention-cnn
Source code for "On the Relationship between Self-Attention and Convolutional Layers"
landmark-attention
Landmark Attention: Random-Access Infinite Context Length for Transformers
collaborative-attention
Code for Multi-Head Attention: Collaborate Instead of Concatenate
error-feedback-SGD
SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847
easy-summary
difficulty-guided text summarization
personalized-collaborative-llms
Exploration on-device self-supervised collaborative fine-tuning of large language models with limited local data availability, using Low-Rank Adaptation (LoRA). We introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based.
cifar
MLO internal cifar 10 / 100 default implementation / reference implementation. single machine, variable batch sizes, allowing maybe gradient compression. need to have clear documentation to make it easy to use, and so that we don't loose time with looking for hyperparameters. we can later keep it in sync with mlbench too, but self-contained is even better
epfml-utils
Tools for experimentation and using run:ai. The aim is for these to be small self-contained utilities that are used by multiple people.