Awesome Federated Machine Learning

Federated Learning (FL) is a new machine learning framework, which enables multiple devices collaboratively to train a shared model without compromising data privacy and security.

This repository will continue to be collected and updated everything about federated learning materials, including research papers, conferences, blogs and beyond.

Top Machine Learning conferences

In this section, we will summarize Federated Learning papers accepted by top machine learning conference, Including Neurips, ICML, ICLR.

ICML

Conferences	Title	Affiliation	Slide & Code
ICML 2021	Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix	Harvard University
	FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Analysis	Peking University; Yale University
	Personalized Federated Learning using Hypernetworks	Bar-Ilan University; NVIDIA Research	code materials
	Federated Composite Optimization	Stanford University; Google	code
	Exploiting Shared Representations for Personalized Federated Learning	University of Texas at Austin; University of Pennsylvania
	Data-Free Knowledge Distillation for Heterogeneous Federated Learning	Michigan State University	code
	Federated Continual Learning with Weighted Inter-client Transfer	KAIST	code
	Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	The University of Iowa
	Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning	The University of Tokyo
	Federated Learning of User Verification Models Without Sharing Embeddings	Qualcomm AI Research
	Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning	Accenture	code
	Ditto: Fair and Robust Federated Learning Through Personalization	CMU; Facebook	code
	Heterogeneity for the Win: One-Shot Federated Clustering	CMU	code
	The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation	Google
	Debiasing Model Updates for Improving Personalized Federated Training	Boston University; Arm Research
	One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning	Toyota Technological Institute of Chicago; University of California, Berkeley; Cornell University
	CRFL: Certifiably Robust Federated Learning against Backdoor Attacks	University of Illinois at Urbana-Champaign; IBM Research; Zhejiang University
	Federated Learning under Arbitrary Communication Patterns	Indiana University, Bloomington; Amazon

ICML 2020	FedBoost: A Communication-Efficient Algorithm for Federated Learning	Google Research	Video
	FetchSGD: Communication-Efficient Federated Learning with Sketching	UC Berkeley; Johns Hopkins University; Amazon	Video Code
	SCAFFOLD: Stochastic Controlled Averaging for Federated Learning	EPFL; Google Research	Video
	Federated Learning with Only Positive Labels	Google Research	Video
	From Local SGD to Local Fixed-Point Methods for Federated Learning	Moscow Institute of Physics and Technology; KAUST	Slide Video
	Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization	KAUST	Slide Video

ICML 2019	Bayesian Nonparametric Federated Learning of Neural Networks	IBM Research	Code
	Analyzing Federated Learning through an Adversarial Lens	Princeton University; IBM Research	Code
	Agnostic Federated Learning	Google Research

ICLR

Conferences	Title	Affiliation	Slide & Code
ICLR 2021	Federated Learning Based on Dynamic Regularization	Boston University; ARM
	Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning	The Ohio State University
	HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients	Duke University	code
	FedMix: Approximation of Mixup under Mean Augmented Federated Learning	KAIST
	Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms	CMU; Google	code
	Adaptive Federated Optimization	Google	code
	Personalized Federated Learning with First Order Model Optimization	Stanford University; NVIDIA
	FedBN: Federated Learning on Non-IID Features via Local Batch Normalization	Princeton University	code
	FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning	The Ohio State University
	Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning	KAIST	code

ICLR 2020	Federated Adversarial Domain Adaptation	Boston University; Columbia University; Rutgers University
	DBA: Distributed Backdoor Attacks against Federated Learning	Zhejiang University; IBM Research	Code
	Fair Resource Allocation in Federated Learning	CMU; Facebook AI	Code
	Federated Learning with Matched Averaging	University of Wisconsin-Madison; IBM Research	Code
	Differentially Private Meta-Learning	CMU
	Generative Models for Effective ML on Private, Decentralized Datasets	Google	Code
	On the Convergence of FedAvg on Non-IID Data	Peking University	Code

NeurIPS

Conferences	Title	Affiliation	Slide & Code
NeurIPS 2020	Differentially-Private Federated Linear Bandits	MIT	code
	Federated Principal Component Analysis	University of Cambridge; Quine Technologies	code
	FedSplit: an algorithmic framework for fast federated optimization	UC Berkeley
	Federated Bayesian Optimization via Thompson Sampling	NUS; MIT
	Lower Bounds and Optimal Algorithms for Personalized Federated Learning	KAUST
	Robust Federated Learning: The Case of Affine Distribution Shifts	UC Santa Barbara; MIT
	An Efficient Framework for Clustered Federated Learning	UC Berkeley; DeepMind	Code
	Distributionally Robust Federated Averaging	Pennsylvania State University	Code
	Personalized Federated Learning with Moreau Envelopes	The University of Sydney	code
	Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach	MIT; UT Austin
	Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge	University of Southern California	code
	Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization	CMU; Princeton University
	Attack of the Tails: Yes, You Really Can Backdoor Federated Learning	University of Wisconsin-Madison
	Federated Accelerated Stochastic Gradient Descent	Stanford University	code
	Inverting Gradients - How easy is it to break privacy in federated learning?	University of Siegen	code
	Ensemble Distillation for Robust Model Fusion in Federated Learning	EPFL
	Throughput-Optimal Topology Design for Cross-Silo Federated Learning	INRIA	code

Others

Conferences	Title	Affiliation	Slide & Code
AISTATS 2020	FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization	UC Santa Barbara; UT Austin	Supplementary
	How To Backdoor Federated Learning	Cornell Tech	Supplementary
	Federated Heavy Hitters Discovery with Differential Privacy	RPI; Google	Supplementary

Books

联邦学习（Federated Learning）

Chinese Version

English Version
联邦学习实战（Practicing Federated Learning）

Chinese Version

Github

Talks and Tutorials

TensorFlow Federated (TFF): Machine Learning on Decentralized Data - Google, TF Dev Summit ‘19 2019
Federated Learning: Machine Learning on Decentralized Data - Google, Google I/O 2019
Federated Learning - Cloudera Fast Forward Labs, DataWorks Summit 2019
GDPR, Data Shortage and AI - Qiang Yang, AAAI 2019 Invited Talk
Making every phone smarter with Federated Learning - Google, 2018

Conferences and Workshops

FL-ICML 2020 - Organized by IBM Watson Research.
FL-IBM 2020 - Organized by IBM Watson Research and Webank.
FL-NeurIPS 2019 - Organized by Google, Webank, NTU, CMU.
FL-IJCAI 2019 - Organized by Webank.
Google Federated Learning workshop - Organized by Google.

Blogs

What is Federated Learning - Nvidia 2019
Online Federated Learning Comic - Google 2019
Federated Learning: Collaborative Machine Learning without Centralized Training Data - Google AI Blog 2017

Papers

1. Personalization

Personalized federated learning refers to train a model for each client, based on the client’s own dataset and the datasets of other clients. There are two major motivations for personalized federated learning：

Due to statistical heterogeneity across clients, a single global model would not be a good choice for all clients. Sometimes, the local models trained solely on their private data perform better than the global shared model.
Different clients need models specifically customized to their own environment. As an example of model heterogeneity, consider the sentence: “I live in .....”. The next-word prediction task applied on this sentence needs to predict a different answer customized for each user. Different clients may assign different labels to the same data.

Personalized federated learning Survey paper:

Methodology	Title	Conferences	Slide & Code
Multi-Task Learning	Federated Multi-Task Learning	Stanford USC CMU
Multi-Task Learning	Variational Federated Multi-Task Learning	ETH Zurich

Meta Learning	Personalized Federated Learning: A Meta-Learning Approach	MIT
	Improving Federated Learning Personalization via Model Agnostic Meta Learning	University of Washington; Google
	Adaptive Gradient-Based Meta-Learning Methods	CMU
	Federated Meta-Learning with Fast Convergence and Efficient Communication	Huawei Noah’s Ark Lab

Mixture of Global and Local Models	Federated Learning of a Mixture of Global and Local Models	KAUST
	Federated User Representation Learning	University of Michigan Facebook
	Adaptive Personalized Federated Learning	The Pennsylvania State University

Personalization Layers	Federated Learning with Personalization Layers	Adobe Research Indian Institute of Technology
Personalization Layers	Think Locally, Act Globally: Federated Learning with Local and Global Representations	CMU University of Tokyo Columbia University

Transfer Learning	Federated evaluation of on-device personalization	Google
	Salvaging Federated Learning by Local Adaptation	Cornell University
	Private Federated Learning with Domain Adaptation	Oracle Labs

Clustering	Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints
Clustering	An Efficient Framework for Clustered Federated Learning	UC Berkeley DeepMind	Code

2. Federated Learning Survey

Advances and Open Problems in Federated Learning - arXiv Dec 2019
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection - arXiv Apr 2020
Federated Learning in Mobile Edge Networks: A Comprehensive Survey - arXiv Sep 2019
Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges - arXiv Sep 2019
Federated Learning: Challenges, Methods, and Future Directions - arXiv Aug 2019
Federated Machine Learning: Concept and Applications - ACM TIST 2018

3. System Design

Towards Federated Learning at Scale: System Design - SysML 2019
A generic framework for privacy preserving deep learning - arXiv 2018
Communication-Efficient Learning of Deep Networks from Decentralized Data - AISTATS 2017 （The first paper proposed federated learning concept）

4. Security

9. Application

9.1. Computer Vision

FedVision: An Online Visual Object Detection Platform Powered by Federated Learning - IAAI 2020

9.2. Nature Language Processing

Federated Topic Modeling - CIKM 2019
Federated Learning for Mobile Keyboard Prediction - arXiv 2019
Applied federated learning: Improving google keyboard query suggestions - arXiv 2018
Federated Learning Of Out-Of-Vocabulary Words - arXiv 2018

9.3. Healthcare

Privacy-preserving Federated Brain Tumour Segmentation - MICCAI MLMI 2019
FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare - arXiv 2019

9.4. Recommendation

Secure Federated Matrix Factorization - arXiv 2019
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System - arXiv 2019
Federated Meta-Learning with Fast Convergence and Efficient Communication - arXiv 2018

9.5. Blockchain

FedCoin: A Peer-to-Peer Payment System for Federated Learning - arXiv Feb 2020
Blockchained On-Device Federated Learning - IEEE Communications Letters 2019

Open-Sources

⬆ Return to top

About

A curated list of awesome federated machine learning resources, including research papers, codes, tutorials, blogs and beyond

chunmeifeng / Awesome-Federated-Machine-Learning