Giters

sleepysss / Awesome-Federated-Machine-Learning

Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Awesome Federated Machine Learning Awesome

Federated Learning (FL) is a new machine learning framework, which enables multiple devices collaboratively to train a shared model without compromising data privacy and security.

FL

This repository aims to keep tracking the latest research advancements of federated learning, including but not limited to research papers, books, codes, tutorials, and videos.

Table of Contents

 

Top Machine Learning Conferences

In this section, we will summarize Federated Learning papers accepted by top machine learning conference, Including NeurIPS, ICML, ICLR.

ICML

Years Title Affiliations Materials
ICML 2022 Deep Neural Network Fusion via Graph Matching with Applications to Model Ensemble and Federated Learning Shanghai Jiao Tong University code
video
Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization KAIST video
slide
FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learning University of Oulu code
video
slide
FedNL: Making Newton-Type Methods Applicable to Federated Learning KAUST video
slide
video
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms Carnegie Mellon University slide
video
FedNest: Federated Bilevel, Minimax, and Compositional Optimization University of Michigan code
video
slide
Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification University of Maryland code
slide
video
DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training University of Science and Technology of China code
video
Federated Learning with Positive and Unlabeled Data Xi’an Jiaotong University video
Neurotoxin: Durable Backdoors in Federated Learning Southeast University;
Princeton University		 code
slide
video
DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning University of Cambridge slide
video
Neural Tangent Kernel Empowered Federated Learning NC State University code
slide
video
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning VMware Research code
slide
video
Architecture Agnostic Federated Learning for Neural Networks The University of Texas at Austin slide
video
Fast Composite Optimization and Statistical Recovery in Federated Learning Shanghai Jiao Tong University slide
video
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning New York University slide
video
Communication-Efficient Adaptive Federated Learning Pennsylvania State University slide
video
Personalized Federated Learning via Variational Bayesian Inference Chinese Academy of Sciences code
video
QSFL: A Two-Level Uplink Communication Optimization Framework for Federated Learning Nankai University code
slide
video
Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy University of Minnesota video
The Poisson Binomial Mechanism for Unbiased Federated Learning with Secure Aggregation Stanford University;
Google Research		 slide
video
The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning Stanford University;
Google Research		 code
video
Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring University of Science and Technology of China slide
video
Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling Geogia Institute of Technology slide
video
Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering University of Michigan code
video
Resilient and Communication Efficient Learning for Heterogeneous Federated Systems Michigan State University slide
video
Accelerated Federated Learning with Decoupled Adaptive Optimization Auburn University slide
video
Proximal and Federated Random Reshuffling KAUST code
video
Personalized Federated Learning through Local Memorization Inria code
video
Federated Learning with Partial Model Personalization University of Washington code
slide
video
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training CISPA Helmholz Center for Information Security code
video
Federated Learning with Label Distribution Skew via Logits Calibration Zhejiang University slide
video
Anarchic Federated Learning The Ohio State University slide
video
Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning Hong Kong Baptist University code
video
Generalized Federated Learning via Sharpness Aware Minimization University of South Florida slide
video
FedScale: Benchmarking Model and System Performance of Federated Learning at Scale University of Michigan code
video
Multi-Level Branched Regularization for Federated Learning Seoul National University HomePage
slide
video
ICML 2021 Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix Harvard University video
code
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Analysis Peking University;
Princeton University		 video
Personalized Federated Learning using Hypernetworks Bar-Ilan University;
NVIDIA 		code
HomePage
video
Federated Composite Optimization Stanford University;
Google		 code
video
slides
Exploiting Shared Representations for Personalized Federated Learning University of Texas at Austin;
University of Pennsylvania		 code
video
Data-Free Knowledge Distillation for Heterogeneous Federated Learning Michigan State University code
video
Federated Continual Learning with Weighted Inter-client Transfer KAIST code
video
Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity The University of Iowa video
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning The University of Tokyo video
Federated Learning of User Verification Models Without Sharing Embeddings Qualcomm video
Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning Accenture code
video
Ditto: Fair and Robust Federated Learning Through Personalization CMU;
Facebook AI		 code
video
Heterogeneity for the Win: One-Shot Federated Clustering CMU video
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation Google video
Debiasing Model Updates for Improving Personalized Federated Training Boston University;
Arm 		video
One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning Toyota;
Berkeley;
Cornell University		 code
video
CRFL: Certifiably Robust Federated Learning against Backdoor Attacks UIUC;
IBM		 code
video
Federated Learning under Arbitrary Communication Patterns Indiana University;
Amazon		 video
ICML 2020 FedBoost: A Communication-Efficient Algorithm for Federated Learning Google Video
FetchSGD: Communication-Efficient Federated Learning with Sketching UC Berkeley;
Johns Hopkins University;
Amazon		 Video
Code
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning EPFL;
Google		 Video
Federated Learning with Only Positive Labels Google Video
From Local SGD to Local Fixed-Point Methods for Federated Learning Moscow Institute of Physics and Technology;
KAUST		 Slide
Video
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization KAUST Slide
Video
ICML 2019 Bayesian Nonparametric Federated Learning of Neural Networks IBM Code
Analyzing Federated Learning through an Adversarial Lens Princeton University;
IBM		 Code
Agnostic Federated Learning Google

ICLR

Years Title Affiliation Materials
ICLR 2023 MocoSFL: enabling cross-client collaborative self-supervised learning Arizona State University Code
Personalized Federated Learning with Feature Alignment and Classifier Collaboration Tsinghua University
Single-shot General Hyper-parameter Optimization for Federated Learning IBM Research
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning Meta AI
FedExP: Speeding Up Federated Averaging via Extrapolation Carnegie Mellon University code
Turning the Curse of Heterogeneity in Federated Learning into a Blessing for Out-of-Distribution Detection Michigan State University code
DASHA: Distributed Nonconvex Optimization with Communication Compression and Optimal Oracle Complexity KAUST code
Federated Neural Bandits National University of Singapore code
Machine Unlearning of Federated Clusters University of Illinois Urbana-Champaign code
FedFA: Federated Feature Augmentation ETH Zurich code
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach Carnegie Mellon University code
Better Generative Replay for Continual Federated Learning University of Virginia code
Federated Learning from Small Datasets University Hospital Essen code
Federated Nearest Neighbor Machine Translation University of Science and Technology of China code
Test-Time Robust Personalization for Federated Learning Westlake University code
DepthFL : Depthwise Federated Learning for Heterogeneous Clients Seoul National University
Towards Addressing Label Skews in One-Shot Federated Learning National University of Singapore code
Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning National University of Singapore code
Bias Propagation in Federated Learning National University of Singapore code
Panning for Gold in Federated Learning: Targeted Text Extraction under Arbitrarily Large-Scale Aggregation University of Maryland
SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication University of Maryland code
Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Losses University of Southern California
Effective passive membership inference attacks in federated learning against overparameterized models Purdue University
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy The University of Sydney
FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification University of Cambridge code
Multimodal Federated Learning via Contrastive Representation Ensemble Tsinghua University
Faster federated optimization under second-order similarity Princeton University
Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning ETH Zurich code
The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation The University of Texas at Austin
PerFedMask: Personalized Federated Learning with Optimized Masking Vectors The University of British Columbia code
FedDAR: Federated Domain-Aware Representation Learning Harvard University code
EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Data George Mason University code
FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning Purdue University code
Generalization Bounds for Federated Learning: Fast Rates, Unparticipating Clients and Unbounded Losses Renmin University of China
Combating Exacerbated Heterogeneity for Robust Models in Federated Learning Hong Kong Baptist University code
Efficient Federated Domain Translation Purdue University
On the Importance and Applicability of Pre-Training for Federated Learning The Ohio State University code
A Statistical Framework for Personalized Federated Learning and Estimation: Theory, Algorithms, and Privacy University of California, Los Angeles
Instance-wise Batch Label Restoration via Gradients in Federated Learning Beihang University code
Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models University of Maryland
Meta Knowledge Condensation for Federated Learning Center for Frontier AI Research
Data-Free One-Shot Federated Learning Under Very High Statistical Heterogeneity William & Mary
CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning University of Warwick code
Sparse Random Networks for Communication-Efficient Federated Learning Stanford University code
Hyperparameter Optimization through Neural Network Partitioning University of Cambridge
Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision? MIT CSAIL code
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top KAUST
ICLR 2022 Bayesian Framework for Gradient Leakage ETH Zurich Code
Federated Learning from only unlabeled data with class-conditional-sharing clients The University of Tokyo;
The Chinese University of Hong Kong		 Code
FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning Carnegie Mellon University;
University of Illinois at Urbana-Champaign;
University of Washington
Acceleration of Federated Learning with Alleviated Forgetting in Local Training Tsinghua University Code
FedPara: Low-rank Hadamard Product for Communication-Efficient Federated Learning POSTECH Code
An Agnostic Approach to Federated Learning with Class Imbalance University of Pennsylvania Code
Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization Michigan State University;
The University of Texas at Austin		 code
Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models University of Maryland;
New York University		 code (Minimum)
code (Comprehensive)
ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity University of Cambridge;
University of Oxford
Diverse Client Selection for Federated Learning via Submodular Maximization Intel;
Carnegie Mellon University		 code
Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank? Purdue University code
Diurnal or Nocturnal? Federated Learning of Multi-branch Networks from Periodically Shifting Distributions University of Maryland;
Google		 code
Towards Model Agnostic Federated Learning Using Knowledge Distillation EPFL
Divergence-aware Federated Self-Supervised Learning Nanyang Technological University;
SenseTime
What Do We Mean by Generalization in Federated Learning? Stanford University;
Google		 code
FedBABU: Toward Enhanced Representation for Federated Image Classification KAIST code
Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing EPFL code
Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters Aibee code
Hybrid Local SGD for Federated Learning with Heterogeneous Communications University of Texas;
Pennsylvania State University
On Bridging Generic and Personalized Federated Learning for Image Classification The Ohio State University code
Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond KAIST;
MIT
ICLR 2021 Federated Learning Based on Dynamic Regularization Boston University;
ARM
Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning The Ohio State University
HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients Duke University code
FedMix: Approximation of Mixup under Mean Augmented Federated Learning KAIST
Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms CMU; Google code
Adaptive Federated Optimization Google code
Personalized Federated Learning with First Order Model Optimization Stanford University; NVIDIA
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization Princeton University code
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning The Ohio State University
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning KAIST code
ICLR 2020 Federated Adversarial Domain Adaptation Boston University;
Columbia University;
Rutgers University
DBA: Distributed Backdoor Attacks against Federated Learning Zhejiang University;
IBM Research		 Code
Fair Resource Allocation in Federated Learning CMU;
Facebook AI		 Code
Federated Learning with Matched Averaging University of Wisconsin-Madison;
IBM Research		 Code
Differentially Private Meta-Learning CMU
Generative Models for Effective ML on Private, Decentralized Datasets Google Code
On the Convergence of FedAvg on Non-IID Data Peking University Code

NeurIPS

Years Title Affiliation Materials
NeurIPS 2022 Federated Learning from Pre-Trained Models: A Contrastive Learning Approach University of Technology Sydney
CalFAT: Calibrated Federated Adversarial Training with Label Skewness Zhejiang University
DENSE: Data-Free One-Shot Federated Learning Zhejiang University
Federated Submodel Optimization for Hot and Cold Data Features Shanghai Jiao Tong University code
SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression CMU code
Factorized-FL: Personalized Federated Learning with Parameter Factorization & Similarity Matching KAIST
FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction Michigan State University code
A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits UCLA code
Byzantine-tolerant federated Gaussian process regression for streaming data Pennsylvania State University
Preservation of the Global Knowledge by Not-True Distillation in Federated Learning KAIST
TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels UC Berkeley code
A Unified Analysis of Federated Learning with Arbitrary Client Participation IBM code
SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training Duke University code
A Communication-efficient Algorithm with Linear Convergence for Federated Minimax Learning Northwestern University
Resource-Adaptive Federated Learning with All-In-One Neural Composition Johns Hopkins University
Fairness in Federated Learning via Core-Stability University of Illinois at Urbana Champaign code
FedSR: A Simple and Effective Domain Generalization Method for Federated Learning University of Oxford code
On Sample Optimality in Personalized Collaborative and Federated Learning Inria
Global Convergence of Federated Learning for Mixed Regression Northeastern University
DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing Hong Kong University of Science and Technology
SAGDA: Achieving Communication Complexity in Federated Min-Max Learning The Ohio State University
SAGDA: Achieving Communication Complexity in Federated Min-Max Learning The Ohio State University
FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning Tsinghua University code
FedPop: A Bayesian Approach for Personalised Federated Learning Skolkovo Institute of Science and Technology code
Self-Aware Personalized Federated Learning Amazon
Recovering Private Text in Federated Learning of Language Models Princeton University code
Communication Efficient Federated Learning for Generalized Linear Bandits University of Virginia code
A Coupled Design of Exploiting Record Similarity for Practical Vertical Federated Learning National University of Singapore code
On Privacy and Personalization in Cross-Silo Federated Learning CMU code
Personalized Online Federated Learning with Multiple Kernels University of California Irvine code
Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with an Inexact Prox KAUST
Learning to Attack Federated Learning: A Model-based Reinforcement Learning Attack Framework Tulane University code
Coresets for Vertical Federated Learning: Regularized Linear Regression and K-Means Clustering Nanjing University code
An Adaptive Kernel Approach to Federated Learning of Heterogeneous Causal Effects National University of Singapore code
LAMP: Extracting Text from Gradients with Language Model Priors ETH Zurich code
SecureFedYJ: a safe feature Gaussianization protocol for Federated Learning Owkin Inc
VF-PS: How to Select Important Participants in Vertical Federated Learning, Efficiently and Securely? Wuhan University
Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning EPFL
Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning KAUST
Taming Fat-Tailed (“Heavier-Tailed” with Potentially Infinite Variance) Noise in Federated Learning The Ohio State University
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning The University of Texas at Austin
Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness Peking University code
On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond Baidu Research
Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams University of Wisconsin-Madison
Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks HKUST
Asymptotic Behaviors of Projected Stochastic Approximation: A Jump Diffusion Perspective Peking University
Subspace Recovery from Heterogeneous Data with Non-isotropic Noise Stanford University
EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization KAUST
On-Demand Sampling: Learning Optimally from Multiple Distributions University of California, Berkeley code
Improved Utility Analysis of Private CountSketch University of Copenhagen code
Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning Huawei Technologies France code
Decentralized Local Stochastic Extra-Gradient for Variational Inequalities Yandex code
BEER: Fast Rate for Decentralized Nonconvex Optimization with Communication Compression Princeton University code
Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning NTT DATA Mathematical Systems Inc
Near-Optimal Collaborative Learning in Bandits Université Paris Cité code
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees Yandex
Towards Optimal Communication Complexity in Distributed Non-Convex Optimization TTIC code
NeurIPS 2021 Sageflow: Robust Federated Learning against Both Stragglers and Adversaries KAIST HomePage
CAFE: Catastrophic Data Leakage in Vertical Federated Learning Rensselaer Polytechnic Institute;
IBM Research		 code
HomePage
Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee NUS code
HomePage
Optimality and Stability in Federated Learning: A Game-theoretic Approach Cornell University code
HomePage
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning UCLA HomePage
The Skellam Mechanism for Differentially Private Federated Learning Google Research;
CMU		 HomePage
No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data NUS;
Huawei		 HomePage
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning University of Minnesota HomePage
Subgraph Federated Learning with Missing Neighbor Generation Emory University;
University of British Columbia;
Lehigh University		 HomePage
Evaluating Gradient Inversion Attacks and Defenses in Federated Learning Princeton University Code
HomePage
Personalized Federated Learning With Gaussian Processes Bar-Ilan University code
HomePage
Differentially Private Federated Bayesian Optimization with Distributed Exploration MIT;
NUS		 code
HomePage
Parameterized Knowledge Transfer for Personalized Federated Learning Hong Kong Polytechnic University;
HomePage
Federated Reconstruction: Partially Local Federated Learning Google Research HomePage
Fast Federated Learning in the Presence of Arbitrary Device Unavailability Tsinghua University;
Princeton University;
MIT		 code
HomePage
FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective Duke University;
Accenture Labs		 code
HomePage
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout KAUST;
Samsung AI Center		 HomePage
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients University of Pennsylvania HomePage
Federated Multi-Task Learning under a Mixture of Distributions INRIA;
Accenture Labs		 code
HomePage
Federated Graph Classification over Non-IID Graphs Emory University HomePage
Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing CMU;
Hewlett Packard Enterprise		 code
HomePage
On Large-Cohort Training for Federated Learning Google;
CMU		 code
HomePage
DeepReduce: A Sparse-tensor Communication Framework for Federated Deep Learning KAUST;
Columbia University;
University of Central Florida		 code
HomePage
PartialFed: Cross-Domain Personalized Federated Learning via Partial Initialization Huawei HomePage
Federated Split Task-Agnostic Vision Transformer for COVID-19 CXR Diagnosis KAIST HomePage
Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning Tsinghua University;
Alibaba;
Weill Cornell Medicine		 code
HomePage
Federated Linear Contextual Bandits The Pennsylvania State University;
Facebook;
University of Virginia		 HomePage
Few-Round Learning for Federated Learning KAIST HomePage
Breaking the centralized barrier for cross-device federated learning EPFL;
Google Research		 code
HomePage
Federated-EM with heterogeneity mitigation and variance reduction Ecole Polytechnique;
Google Research		 HomePage
Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning MIT;
Amazon;
Google		 HomePage
FedDR – Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization University of North Carolina at Chapel Hill;
IBM Research		 code
HomePage
Gradient Inversion with Generative Image Prior Pohang University of Science and Technology;
University of Wisconsin-Madison;
University of Washington		 code
HomePage
NeurIPS 2020 Differentially-Private Federated Linear Bandits MIT code
Federated Principal Component Analysis University of Cambridge;
Quine Technologies		 code
FedSplit: an algorithmic framework for fast federated optimization UC Berkeley
Federated Bayesian Optimization via Thompson Sampling NUS; MIT
Lower Bounds and Optimal Algorithms for Personalized Federated Learning KAUST
Robust Federated Learning: The Case of Affine Distribution Shifts UC Santa Barbara; MIT
An Efficient Framework for Clustered Federated Learning UC Berkeley; DeepMind Code
Distributionally Robust Federated Averaging Pennsylvania State University Code
Personalized Federated Learning with Moreau Envelopes The University of Sydney code
Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach MIT; UT Austin
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge University of Southern California code
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization CMU;
Princeton University
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning University of Wisconsin-Madison
Federated Accelerated Stochastic Gradient Descent Stanford University code
Inverting Gradients - How easy is it to break privacy in federated learning? University of Siegen code
Ensemble Distillation for Robust Model Fusion in Federated Learning EPFL
Throughput-Optimal Topology Design for Cross-Silo Federated Learning INRIA code
NeurIPS 2018 cpSGD: Communication-efficient and differentially-private distributed SGD Princeton University;
Google
NeurIPS 2017 Federated Multi-Task Learning Stanford;
USC;
CMU		 code

 

Top Computer Vision Conferences

In this section, we will summarize Federated Learning papers accepted by top computer vision conference, Including CVPR, ICCV, ECCV.

CVPR

Years Title Affiliation Materials
CVPR 2022 FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction National University of Defense Technology code
Federated Class-Incremental Learning Chinese Academy of Sciences;
Northwestern University;
University of Technology Sydney		 code
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning Peking University;
JD Explore Academy;
The University of Sydney
Differentially Private Federated Learning with Local Regularization and Sparsification Chinese Academy of Sciences
Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage University of Tennessee;
Oak Ridge National Laboratory;
Google Research		 code
video
FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learning Duke University
Learn from Others and Be Yourself in Heterogeneous Federated Learning Wuhan University code
video
ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning Arizona State University code
Robust Federated Learning with Noisy and Heterogeneous Clients Wuhan University code
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning Stanford University video
code
CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning Shanghai Jiao Tong University
FedCorr: Multi-Stage Federated Learning for Label Noise Correction Singapore University of Technology and Design video
code
ATPFL: Automatic Trajectory Prediction Model Design under Federated Learning Framework Harbin Institute of Technology
Federated Learning with Position-Aware Neurons Nanjing University
RSCFed: Random Sampling Consensus Federated Semi-supervised Learning The Hong Kong University of Science and Technology code
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation Univ. of Pittsburgh;
NVIDIA		 code
Layer-wised Model Aggregation for Personalized Federated Learning The Hong Kong Polytechnic University
Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning University of Central Florida code
CVPR 2021 Multi-Institutional Collaborations for Improving Deep Learning-Based Magnetic Resonance Image Reconstruction Using Federated Learning Johns Hopkins University code
Model-Contrastive Federated Learning National University of Singapore;
UC Berkeley		 code
FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space The Chinese University of Hong Kong code
Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective Duke University code

ECCV

Years Title Affiliation Materials
ECCV 2022 Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation Johns Hopkins University code
Personalizing Federated Medical Image Segmentation via Local Calibration Xiamen Universit code
SphereFed: Hyperspherical Federated Learning Harvard University
FedX: Unsupervised Federated Learning with Cross Knowledge Distillation KAIST code
FedVLN: Privacy-preserving Federated Vision-and-Language Navigation University of California code
Addressing Heterogeneity in Federated Learning via Distributional Transformation Johns Hopkins University code
FedLTN: Federated Learning for Sparse and Personalized Lottery Ticket Networks MIT
Improving Generalization in Federated Learning by Seeking Flat Minima Politecnico di Torino code
Federated Self-supervised Learning for Video Understanding TCL AI Lab code
AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation Dalhousie University
ECCV 2020 Federated Visual Classification with Real-World Data Distribution MIT;
Google		 Video

ICCV

Years Title Affiliation Materials
ICCV 2021 Federated Learning for Non-IID Data via Unified Feature Learning and Optimization Objective Alignment Peking University
Ensemble Attention Distillation for Privacy-Preserving Federated Learning University at Buffalo
Collaborative Unsupervised Visual Representation Learning from Decentralized Data Nanyang Technological University;
SenseTime

 

Top Artificial Intelligence and Data Mining Conferences

In this section, we will summarize Federated Learning papers accepted by top AI and DM conference, Including AAAI, AISTATS, KDD.

AAAI

Years Title Affiliation Materials
AAAI 2022 HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images The Chinese University of Hong Kong;
Beihang University		 code
Cross-Modal Federated Human Activity Recognition via Modality-Agnostic and Modality-Specific Representation Learning Chinese Academy of Sciences
FedInv: Byzantine-Robust Federated Learning by Inversing Local Model Updates Nanjing University of Aeronautics and Astronautics
Learning Advanced Client Selection Strategy for Federated Learning Harvard University
Federated Learning for Face Recognition with Gradient Correction Beijing University of Posts and Telecommunications
SpreadGNN: Decentralized Multi-Task Federated Learning for Graph Neural Networks on Molecular Data university of Southern California code
SmartIdx: Reducing Communication Cost in Federated Learning by Exploiting the CNNs Structures Harbin Institute of Technology;
Peng Cheng Laboratory
Is Your Data Relevant?: Dynamic Selection of Relevant Data for Federated Learning IIT Bombay
Seizing Critical Learning Periods in Federated Learning SUNY-Binghamton University;
Louisiana State University
Coordinating Momenta for Cross-silo Federated Learning University of Pittsburgh
FedProto: Federated Prototype Learning over Heterogeneous Devices University of Technology Sydney;
University of Washington		 code
FedSoft: Soft Clustered Federated Learning with Proximal Local Updating Carnegie Mellon University
Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better The University of Texas at Austin code
FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition National Taiwan University code
SplitFed: When Federated Learning Meets Split Learning CSIRO;
Lehigh University		 code
Efficient Device Scheduling with Multi-Job Federated Learning Soochow University;
Baidu
Implicit Gradient Alignment in Distributed and Federated Learning IIT Kanpur;
EPFL
Federated Nearest Neighbor Classification with a Colony of Fruit-Flies IBM Research;
Wichita State University		 code
AAAI 2021 Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating Xidian University;
JD Tech		 video
FedRec++: Lossless Federated Recommendation with Explicit Feedback Shenzhen University video
Federated Multi-Armed Bandits University of Virginia code
video
On the Convergence of Communication-Efficient Local SGD for Federated Learning Temple University;
University of Pittsburgh		 video
FLAME: Differentially Private Federated Learning in the Shuffle Model Renmin University of China;
Kyoto University		 video
code
Toward Understanding the Influence of Individual Clients in Federated Learning Shanghai Jiao Tong University;
The University of Texas at Dallas		 video
Provably Secure Federated Learning against Malicious Clients Duke University video
slides
Personalized Cross-Silo Federated Learning on Non-IID Data Simon Fraser University;
McMaster University		 video
Model-Sharing Games: Analyzing Federated Learning under Voluntary Participation Cornell University code
video
Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning University of Nevada;
IBM Research		 video
Game of Gradients: Mitigating Irrelevant Clients in Federated Learning IIT Bombay;
IBM Research		 video
Supplementary
Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models The Chinese University of Hong Kong;
Arizona State University		 video
code
Adressing Class Imbalance in Federated Learning Northwestern University video
code
Defending against Backdoors in Federated Learning with Robust Learning Rate The University of Texas at Dallas video
code
AAAI 2020 Practical Federated Gradient Boosting Decision Trees National University of Singapore;
The University of Western Australia		 code
Federated Learning for Vision-and-Language Grounding Problems Peking University;
Tencent
Federated Latent Dirichlet Allocation: A Local Differential Privacy Based Framework Beihang University
Federated Patient Hashing Cornell University
Robust Federated Learning via Collaborative Machine Teaching Symantec Research Labs;
KAUST

AISTATS

Years Title Affiliation Materials
AISTATS 2022 Federated Reinforcement Learning with Environment Heterogeneity Peking University code
Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning KAUST
Federated Learning with Buffered Asynchronous Aggregation Meta AI video
Federated Myopic Community Detection with One-shot Communication Purdue University
QLSD: Quantised Langevin Stochastic Dynamics for Bayesian Federated Learning Criteo AI Lab video
code
Asynchronous Upper Confidence Bound Algorithms for Federated Linear Bandits University of Virginia code
SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification Princeton University video
code
Federated Functional Gradient Boosting University of Pennsylvania code
Towards Federated Bayesian Network Structure Learning with Continuous Optimization Carnegie Mellon University code
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Stanford University code
Differentially Private Federated Learning on Heterogeneous Data Stanford University code
Towards Understanding Biased Client Selection in Federated Learning Carnegie Mellon University code
FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning KAUST code
AISTATS 2021 Free-rider Attacks on Model Aggregation in Federated Learning Accenture Labs video
Supplementary
Federated f-differential privacy University of Pennsylvania code
video
Supplementary
Federated learning with compression: Unified analysis and sharp guarantees The Pennsylvania State University;
The University of Texas at Austin		 code
video
Supplementary
Shuffled Model of Differential Privacy in Federated Learning UCLA;
Google		 video
Supplementary
Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning Google video
Supplementary
Federated Multi-armed Bandits with Personalization University of Virginia;
The Pennsylvania State University		 code
video
Supplementary
Towards Flexible Device Participation in Federated Learning CMU;
Sun Yat-Sen University		 video
Supplementary
AISTATS 2020 FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization UC Santa Barbara;
UT Austin		 video
Supplementary
How To Backdoor Federated Learning Cornell Tech video
code
Supplementary
Federated Heavy Hitters Discovery with Differential Privacy RPI;
Google		 video
Supplementary

KDD

Years Sessions Title Affiliation Materials
KDD 2022 Research Track Collaboration Equilibrium in Federated Learning Tsinghua University;
Alibaba Group		 code
Connected Low-Loss Subspace Learning for a Personalization in Federated Learning Ulsan National Institute of Science and Technology & Kakao Enterprise code
Communication-Efficient Robust Federated Learning with Noisy Labels University of Pittsburgh
FedMSplit: Correlation-Adaptive Federated Multi-Task Learning across Multimodal Split Networks
Application Track Fed-LTD: Towards Cross-Platform Ride Hailing via Federated Learning to Dispatch Beihang University
No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices Renmin University of China
EasyFGL: Towards a Unified, Comprehensive and Efficient Platform for Federated Graph Learning Alibaba Group code
FedAttack: Effective and Covert Poisoning Attack on Federated Recommendation via Hard Sampling Tsinghua University code
KDD 2021 Research Track Fed2: Feature-Aligned Federated Learning George Mason University;
Microsoft;
University of Maryland
FedRS: Federated Learning with Restricted Softmax for Label Distribution Non-IID Data Nanjing University code
Federated Adversarial Debiasing for Fair and Trasnferable Representations Michigan State University HomePage
Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling University of Southern California code
Application Track AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization JD Tech
FLOP: Federated Learning on Medical Datasets using Partial Networks Duke University code
KDD 2020 Research Track FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems University College Dublin video
Application Track Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data JD Tech video

 

Books

Papers

1. Model Aggregation

Model Aggregation (or Model Fusion) refers to how to combine local models into a shared global model.

Papers Abbreviation Conferences/Affiliations Materials
Communication-Efficient Learning of Deep Networks from Decentralized Data FedAvg ASTATS 2017
Bayesian Nonparametric Federated Learning of Neural Networks PFNM ICML 2019 code
Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent Krum NeurIPS 2017
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates median;
trimmed mean		 ICML 2018
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms median;
mean		 NeurIPS 2020
The hidden vulnerability of distributed learning in byzantium Bulyan ICML 2018
Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance Zeno ICML 2019 code
Statistical Model Aggregation via Parameter Matching SPAHM NeurIPS 2019 code
Fed+: A Unified Approach to Robust Personalized Federated Learning Fed+
FEDERATED OPTIMIZATION IN HETEROGENEOUS NETWORKS FedProx MLSys 2020 code
Separation of Powers in Federated Learning Truda
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning FedBE ICLR 2021
(The Ohio State University)
Federated Learning with Matched Averaging FedMA ICLR 2020
(University of Wisconsin-Madison; IBM)		 Code
FedSim: Similarity Guided Model Aggregation for Federated Learning FedSim Neurocomputing 2022 (Vol 483)
(Robert Gordon University, UK)		 Code
Model Fusion via Optimal Transport NeurIPS 2020
(ETH, EPFL)		 Code

 

2. Personalization

Personalized federated learning refers to train a model for each client, based on the client’s own dataset and the datasets of other clients. There are two major motivations for personalized federated learning：

  • Due to statistical heterogeneity across clients, a single global model would not be a good choice for all clients. Sometimes, the local models trained solely on their private data perform better than the global shared model.
  • Different clients need models specifically customized to their own environment. As an example of model heterogeneity, consider the sentence: “I live in .....”. The next-word prediction task applied on this sentence needs to predict a different answer customized for each user. Different clients may assign different labels to the same data.

Personalized federated learning Survey paper:

Methodology Papers Conferences/Affiliations Materials
Multi-Task Learning Federated Multi-Task Learning NeurIPS 2017
(Stanford; USC; CMU)		 code
Decentralized Collaborative Learning of Personalized Models over Networks AISTATS 2017
(INRIA)
Variational Federated Multi-Task Learning ETH Zurich
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs AISTATS 2020
(INRIA)		 video
Personalized Cross-Silo Federated Learning on Non-IID Data AAAI 2021
(Simon Fraser University; McMaster University; Huawei Technologies Canada)		 video
Ditto: Fair and Robust Federated Learning Through Personalization ICML 2021
(CMU; Facebook AI)		 code
video
Federated Multi-Task Learning under a Mixture of Distributions NeurIPS 2021
(Inria; Accenture Labs)		 code
Meta Learning Personalized Federated Learning: A Meta-Learning Approach MIT
Debiasing Model Updates for Improving Personalized Federated Training ICML 2021
(Boston University; Arm) 		video
Improving Federated Learning Personalization via Model Agnostic Meta Learning University of Washington;
Google
Adaptive Gradient-Based Meta-Learning Methods CMU
Federated Meta-Learning with Fast Convergence and Efficient Communication Huawei Noah’s Ark Lab
Mixture of Global and Local Models Federated Learning of a Mixture of Global and Local Models KAUST
Federated User Representation Learning University of Michigan
Facebook
Adaptive Personalized Federated Learning The Pennsylvania State University
Personalization Layers Federated Learning with Personalization Layers Adobe Research
Indian Institute of Technology
Think Locally, Act Globally: Federated Learning with Local and Global Representations CMU
University of Tokyo
Columbia University
Exploiting Shared Representations for Personalized Federated Learning ICML 2021
(University of Texas at Austin;
University of Pennsylvania)		 code
video
Transfer Learning Federated evaluation of on-device personalization Google
Salvaging Federated Learning by Local Adaptation Cornell University
Private Federated Learning with Domain Adaptation Oracle Labs
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning NeurIPS 2021
(UCLA)		 HomePage
Clustering Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints Fraunhofer Heinrich Hertz Institute Code
An Efficient Framework for Clustered Federated Learning UC Berkeley
DeepMind		 Code
Robust Federated Learning in a Heterogeneous Environment UC Berkeley
Personalized Federated Learning with First Order Model Optimization ICLR 2021
(Stanford University; NVIDIA)
Hypernetwork Personalized Federated Learning using Hypernetworks Bar-Ilan University;
NVIDIA 		code
HomePage
video

 

3. Recommender system

Recommender system (RecSys) is widely used to solve information overload. In general, the more data RecSys use, the better the recommendation performance we can obtain.

Traditionally, RecSys requires the data that are distributed across multiple devices to be uploaded to the central database for model training. However, due to privacy and security concerns, such directly sharing user data strategies are no longer appropriate.

The incorporation of federated learning and RecSys is a promising approach, which can alleviate the risk of privacy leakage.

More federated recommendation papers can be found in this repository: FedRecPapers

Methodology Papers Conferences/Affiliations Materials
Matrix Factorization Secure federated matrix factorization IEEE Intelligent Systems
Federated Multi-view Matrix Factorization for Personalized Recommendations ECML-PKDD 2020 video
Decentralized Recommendation Based on Matrix Factorization: A Comparison of Gossip and Federated Learning ECML-PKDD 2019
Towards Privacy-preserving Mobile Applications with Federated Learning: The Case of Matrix Factorization MobiSys 2019
Meta Matrix Factorization for Federated Rating Predictions ACM SIGIR 2020 code
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System Arxiv
GNN FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation Arxiv
Federated Social Recommendation with Graph Neural Network ACM TIST
Personalized Federated Recommendation via Joint Representation Learning, User Clustering, and Model Adaptation CIKM 2022
Federated Meta Embedding Concept Stock Recommendation IEEE Big Data 2022
Federated Learning-Based Cross-Enterprise Recommendation With Graph Neural Networks IEEE Transactions on Industrial Informatics

 

4. Security

4.1. Attack

Methodology Papers Conferences/Affiliations Materials
Backdoor Attack How To Backdoor Federated Learning AISTATS 2020 code
Can You Really Backdoor Federated Learning? Arxiv
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning NeurIPS 2020 code
DBA: Distributed Backdoor Attacks against Federated Learning ICLR 2020 code
Gradients Attack Deep Leakage from Gradients NeurIPS 2020 HomePage
code
Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix ICML 2021 code
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning ACM CCS 2017 video
iDLG: Improved Deep Leakage from Gradients Arxiv code
See through Gradients: Image Batch Recovery via GradInversion CVPR 2021
Inverting Gradients - How easy is it to break Privacy in Federated Learning? NeurIPS 2020 code
CAFE: Catastrophic Data Leakage in Vertical Federated Learning NeurIPS 2021
(Rensselaer Polytechnic Institute; IBM)		 code
HomePage
Evaluating Gradient Inversion Attacks and Defenses in Federated Learning NeurIPS 2021
(Princeton University)
R-GAP: RECURSIVE GRADIENT ATTACK ON PRIVACY ICLR 2021
(KU Leuven, Belgium)		 Code
Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage CVPR 2022
(University of Tennessee;Oak Ridge National Laboratory)		 Code
Model Poison Attack Analyzing Federated Learning through an Adversarial Lens ICML 2019
(Princeton University; IBM)		 Code

 

4.2. Defense

Methodology Papers Conferences/Affiliations Materials
FL+DP Federated Learning With Differential Privacy: Algorithms and Performance Analysis IEEE Transactions on Information Forensics and Security
Differentially Private Federated Learning: A Client Level Perspective Arxiv code
Learning Differentially Private Recurrent Language Models ICLR 2018
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation ICML 2021
(Google)		 video
The Skellam Mechanism for Differentially Private Federated Learning NeurIPS 2021
(Google Research; CMU)		 HomePage
FL+HE Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption Arxiv
BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning USENIX 2020 code
FL+TEE PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments ACM MobiSys 2021
Darknetz: towards model privacy at the edge using trusted execution environments. ACM MobiSys 2020 code
video
Algorithm A Little Is Enough: Circumventing Defenses For Distributed Learning NeurIPS 2019
CRFL: Certifiably Robust Federated Learning against Backdoor Attacks ICML 2021
(UIUC; IBM)		 code
video
Evaluating Gradient Inversion Attacks and Defenses in Federated Learning NeurIPS 2021
(Princeton University)		 Code
HomePage

 

5. Survey

Category Papers
General Federated Learning with Privacy-preserving and Model IP-right-protection
Federated machine learning: Concept and applications
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Advances and Open Problems in Federated Learning
Federated Learning: Challenges, Methods, and Future Directions
The Internet of Federated Things (IoFT)
Vertical Federated Learning: Challenges, Methodologies and Experiments
Vertical Federated Learning
Security A survey on security and privacy of federated learning
Threats to Federated Learning: A Survey
Vulnerabilities in Federated Learning
Personalization Survey of Personalization Techniques for Federated Learning
Towards Personalized Federated Learning
Aggregation Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
Incentive A Comprehensive Survey of Incentive Mechanism for Federated Learning
A Survey of Incentive Mechanism Design for Federated Learning
Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspective
Applications A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
Fairness A Survey of Fairness-Aware Federated Learning
Graph FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks
Federated Graph Learning - A Position Paper
Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applications
Federated Graph Neural Networks: Overview, Techniques and Challenges
System Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies

 

7. Efficiency

Communication-Based: Improving efficiency by reducing model parameters transmission.

Hardware-Based: Improving efficiency by hardware acceleration (GPU, FPGA, etc.)

Algorithm-Based: Improving efficiency by accelerating model convergence rate (local training, model aggregation, client selection, etc.)

Taxonomy Papers Techniques Materials
Communication-Based Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients Quantization
Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning Quantization
Communication Efficient Federated Learning with Adaptive Quantization Quantization
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding Quantization
SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communications Sparsity
RPN: A Residual Pooling Network for Efficient Federated Learning Sparsity
FedMD: Heterogenous Federated Learning via Model Distillation Knowledge Distillation code
Ensemble distillation for robust model fusion in federated learning Knowledge Distillation code
Hardware-Based FPGA-Based Hardware Accelerator of Homomorphic Encryption for Efficient Federated Learning FPGA-Based Acceleration
Hardware Accelerated Learning at the Edge GPU-Based Acceleration
HAFLO: GPU-Based Acceleration for Federated Logistic Regression GPU-Based Acceleration
Algorithm FetchSGD: Communication-Efficient Federated Learning with Sketching Optimization Video
Code
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization Optimization Slide
Video
FedBoost: A Communication-Efficient Algorithm for Federated Learning Optimization Video
Federated Learning: Strategies for Improving Communication Efficiency Optimization
One-Shot Federated Learning Model Aggregation

 

8. Optimization

Papers Application Scenarios Conferences/Affiliations Materials
Federated Composite Optimization loss function contains a non-smooth regularizer ICML 2021(Google) code
Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity Federated Deep AUC Maximization ICML 2021
(The University of Iowa)		 video
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning Non-Convex Objective optimization ICML 2021
(The University of Tokyo)		 video
From Local SGD to Local Fixed-Point Methods for Federated Learning fixed-point algorithms optimization ICML 2020
(Moscow Institute of Physics and Technology; KAUST)		 Slide
Video
Federated Learning Based on Dynamic Regularization In each round, the objective function for each device dynamically updates its regularizer ICLR 2021
(Boston University; ARM)
Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms Formulate federated learning optimization as a posterior inference problem ICLR 2021
(CMU; Google)		 code
Adaptive Federated Optimization Federated versions of adaptive optimizers ICLR 2021
(Google)		 code
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization How to uses local batch normalization to alleviate the feature shift before averaging models. ICLR 2021
(Princeton University)		 code
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning SFederated versions of emi-Supervised Learning ICLR 2021
(KAIST)		 code
Sageflow: Robust Federated Learning against Both Stragglers and Adversaries Handle both stragglers (slow devices) and adversaries simultaneously NeurIPS 2021
(KAIST)		 HomePage
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning Distributed stochastic non-convex optimization University of Minnesota HomePage
Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating Vertical Federated Learning Optimization AAAI 2021
(Xidian University; JD Tech)		 video

 

9. Fairness

Taxonomy Papers Conferences/Affiliations Materials
Performance Fairness Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning NeurIPS 2021
(Tsinghua University; Alibaba)		 code
HomePage
Fairness-aware Agnostic Federated Learning SDM 2021
(University of Arkansas)
Fair Resource Allocation in Federated Learning ICLR 2020
(CMU; Facebook AI)		 Code
Agnostic Federated Learning ICML 2019
(Google)
Mitigating Bias in Federated Learning arXiv
(IBM)
Ditto: Fair and Robust Federated Learning Through Personalization ICML 2021
(CMU; Facebook AI)		 code
video
Client Selection Fairness An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee IEEE Transactions on Parallel and Distributed Systems
Stochastic Client Selection for Federated Learning with Volatile Clients IEEE Internet of Things Journal
Federated learning with class imbalance reduction European Signal Processing Conference
Reputation-Based Federated Learning for Secure Wireless Networks ” IEEE Internet of Things Journal
Contribution Fairness Profit Allocation for Federated Learning 2019 IEEE International Conference on Big Data
Stochastic Client Selection for Federated Learning with Volatile Clients IEEE Internet of Things Journal
Federated learning with class imbalance reduction European Signal Processing Conference
Reputation-Based Federated Learning for Secure Wireless Networks ” IEEE Internet of Things Journal

 

10. Applications

Applications Papers Conferences/Affiliations Materials
Computer Vision FedVision: An Online Visual Object Detection Platform Powered by Federated Learning WeBank (AAAI 2020) code
Nature Language Processing Federated learning for emoji prediction in a mobile keyboard Google
Federated Learning for Mobile Keyboard Prediction Google
Applied federated learning: Improving google keyboard query suggestions Google
Federated Learning Of Out-Of-Vocabulary Words Google
Automatic Speech Recognition A Federated Approach in Training Acoustic Models MicroSoft (INTERSPEECH 2020) Video
Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion? INRIA (INTERSPEECH 2019)
Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework Google (ICASSP 2021) Google Assistant Help
Federated Evaluation and Tuning for On-Device Personalization: System Design \& Applications Apple Report
Healthcare Privacy-preserving Federated Brain Tumour Segmentation NVIDIA (MICCAI MLMI 2019)
Advancing health research with Google Health Studies Google Blog
Multi-institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation Intel Blog
Blockchain FedCoin: A Peer-to-Peer Payment System for Federated Learning Arxiv
Blockchained On-Device Federated Learning IEEE Communications Letters 2019

 

11. Boosting

Category Papers Conferences/Affiliations Materials
Tree-Base Boosting Practical Federated Gradient Boosting Decision Trees AAAI 2020
(NUS)		 code
Secureboost: A lossless federated learning framework IEEE Intelligent Systems 2021
(WeBank; HKUST)
Large-scale Secure XGB for Vertical Federated Learning CIKM 2021
(Ant Group)		 video

 

12. Incentive mechanism

Typically, the incentive mechanism consists of the following two steps:

  • How to evaluate the contribution of each participant (Shapley value)

  • How to allocate profits based on contributions

Steps Techniques Papers Affiliations Materials
1. Contribution Evaluation Shapley Value Data Shapley: Equitable Valuation of Data for Machine Learning ICML 2019
(Stanford University)		 code
A principled approach to data valuation for federated learning Arxiv
(Harvard University)
Measure contribution of participants in federated learning IEEE Big Data
(Swiss Re)
GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning IAAI 2021
(NTU)
Profit allocation for federated learning Arxiv
(Beihang University)
Fedcoin: A peer-to-peer payment system for federated learning NTU
2. Profit Allocation Contract Theory Hierarchical Incentive Mechanism Design for Federated Machine Learning in Mobile networks IEEE Internet of Things Journal 2020
Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach 2019 IEEE VTS Asia Pacific Wireless Communications Symposium
Incentive Mechanism for Reliable Federated Learning: A Joint Optimization Approach to Combining Reputation and Contract Theory IEEE Internet of Things Journal 2019
A Contract Theory based Incentive Mechanism for Federated Learning FTL-IJCAI Workshop 2021
Stackelberg Game Motivating Workers in Federated Learning: A Stackelberg Game Perspective IEEE Networking Letters, 2020.
A Learning-based Incentive Mechanism for Federated Learning IEEE Internet of Things Journal, 2020
Auction FMore: An Incentive Scheme of Multi-dimensional Auction for Federated Learning in MEC IEEE ICDCS, 2020.
A VCG-based Fair Incentive Mechanism for Federated Learning arXiv preprint
Toward an Automated Auction Framework for Wireless Federated Learning Services Market IEEE Transactions on Mobile Computing, 2020
Auction based Incentive Design for Efficient Federated Learning in Cellular Wireless Networks 2020 IEEE Wireless Communications and Networking Conference
FAIR: Quality-Aware Federated Learning with Precise User Incentive and Model Aggregation IEEE INFOCOM, 2021.
Incentive Mechanism for Horizontal Federated Learning Based on Reputation and Reverse Auction WWW, 2021 video

 

13. Unsupervised Learning

Category Papers Conferences/Affiliations Materials
Clustering Heterogeneity for the Win: One-Shot Federated Clustering ICML 2021
(CMU)		 video
Representations Learning Exploiting Shared Representations for Personalized Federated Learning ICML 2021
(University of Texas at Austin;
University of Pennsylvania)		 code
video
Towards Federated Unsupervised Representation Learning EdgeSys '20
(Eindhoven University of Technology)
Federated Unsupervised Representation Learning Zhejiang University
Collaborative Unsupervised Visual Representation Learning from Decentralized Data (ICCV 2021)
(Nanyang Technological University;SenseTime)
Divergence-aware Federated Self-Supervised Learning (ICLR 2022)
(Nanyang Technological University)
Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering (ICML 2022)
(University of Michigan;University of Cambridge)		 code

 

14. Heterogeneity

Category Papers Conferences/Affiliations Materials
Data Heterogeneity (NON-IID) Data-Free Knowledge Distillation for Heterogeneous Federated Learning ICML 2021
(Michigan State University)		 code
video
Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity ICML 2021
(The University of Iowa)		 video
Exploiting Shared Representations for Personalized Federated Learning ICML 2021
(University of Texas at Austin;
University of Pennsylvania)		 code
video
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning ICML 2020
(EPFL; Google)		 Video
Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning ICLR 2021
(The Ohio State University)
HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients ICLR 2021
(Duke University)		 code
FedMix: Approximation of Mixup under Mean Augmented Federated Learning ICLR 2021
(KAIST)
On the Convergence of FedAvg on Non-IID Data ICLR 2020
(Peking University)		 Code
No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data NeurIPS 2021
(NUS; Huawei)		 HomePage
Model Heterogeneity Personalized Federated Learning with Moreau Envelopes NeurIPS 2020
(The University of Sydney)		 code
Federated Learning of a Mixture of Global and Local Models KAUST
Salvaging Federated Learning by Local Adaptation Cornell University
Device Heterogeneity Sageflow: Robust Federated Learning against Both Stragglers and Adversaries NeurIPS 2021
(KAIST)		 HomePage
Towards Flexible Device Participation in Federated Learning AISTATS 2021
(CMU; Sun Yat-Sen University)		 video
Supplementary
Asynchronous Federated Optimization 12th Annual Workshop on Optimization for Machine Learning
Pisces: Efficient Federated Learning via Guided Asynchronous Training ACM SoCC 2022
(HKUST)		 Code
Slides

 

15. Client Selection

Papers Conferences/Affiliations Materials
Diverse Client Selection for Federated Learning via Submodular Maximization ICLR 2022
(Intel; CMU)		 code
TiFL: A Tier-based Federated Learning System HPDC 2020
(George Mason University)
Communication-Efficient Federated Learning via Optimal Client Sampling University of Texas at Austin
Oort: Efficient Federated Learning via Guided Participant Selection OSDI 2021
(University of Michigan)		 code
Optimizing federated learning on non-iid data with reinforcement learning IEEE INFOCOM 2020
(University of Toronto)
Learning Advanced Client Selection Strategy for Federated Learning AAAI 2022
(Harvard University)
Towards understanding biased client selection in federated learning AISTATS 2022
(Carnegie Mellon University)
Stochastic Client Selection for Federated Learning with Volatile Clients IEEE Internet of Things Journal 2022
(South China University of Technology)
Oort: Efficient Federated Learning via Guided Participant Selection OSDI 2021
(University of Michigan)		 code
video
Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning ICML 2021
(Accenture)		 code
video
Federated Multi-Armed Bandits AAAI 2021
(University of Virginia)		 code
video
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization AISTATS 2020
(UC Santa Barbara; UT Austin)		 video
Supplementary

 

16. Graph Neural Networks

Category Papers Conferences/Affiliations Materials
Graph-Level Federated Graph Classification over Non-IID Graphs NeurIPS 2021
(Emory University)
Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting arXiv
(University of Rochester)
FedGCN: Federated Learning-Based Graph Convolutional Networks for Non-Euclidean Spatial Data Mathematics 2022
(Nanjing University of Information Science and Technology)
SubGraph-Level FedGraph: Federated Graph Learning with Intelligent Sampling IEEE TPDS
(University of Aizu)
Glint: Decentralized Federated Graph Learning with Traffic Throttling and Flow Scheduling IWQOS 2021
(University of Aizu)
PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN Training Method Arxiv
(Xi’an Jiaotong University)
Subgraph Federated Learning with Missing Neighbor Generation NeurIPS 2021
(Emory University;)		 HomePage
FedNI: Federated Graph Learning with Network Inpainting for Population-Based Disease Prediction Arxiv
( University of Electronic Science and Technology of China)
FedGL: Federated Graph Learning Framework with Global Self-Supervision Arxiv
(Sun Yat-sen University)
Node-Level Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling KDD 2021
(University of Southern California)
Personalized Federated Learning With Graph IJCAI 2022
(Beihang University)		 code
BiG-Fed: Bilevel Optimization Enhanced Graph-Aided Federated Learning FL-ICML Workshop 2021
(Nanyang Technological University)
Decentralized Federated Graph Neural Networks FL-IJCAI Workshop 2021
(Blue Elephant Tec)
Peer-to-Peer Federated Learning on Graphs Arxiv
(University of California)
A Graph Federated Architecture with Privacy Preserving Learning Arxiv
(EPFL)
Decentralized Federated Learning for Electronic Health Records Arxiv
(IBM)
Learn Electronic Health Records by Fully Decentralized Federated Learning FL-NeurIPS 2019
(University of Minnesota)
Decentralized Federated Learning via SGD over Wireless D2D Networks Arxiv
(Shenzhen University)
SpreadGNN: Decentralized Multi-Task Federated Learning for Graph Neural Networks on Molecular Data AAAI 2022
(university of Southern California)		 code
A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularization Arxiv
(The University of Sydney)		 code
Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds Arxiv
( Purdue University)		 code

 

18. Other Machine Learning Paradigm

Taxonomy Papers Conferences/Affiliations Materials
Neural Architecture Search Federated Neural Architecture Search Beijing University of Posts and Telecommunications
FedNAS: Federated Deep Learning via Neural Architecture Search CVPR 2020 workshop
(University of Southern California)
Self-supervised cross-silo federated neural architecture search WeBank
Active Learning Active Federated Learning University of Michigan
Active learning based federated learning for waste and natural disaster image classification Hamad Bin Khalifa University
Federated Active Learning Harvard Medical School
Continual Learning Federated Continual Learning with Weighted Inter-client Transfer KAIST (ICML 2021) code
Partitioned variational inference: A unified framework encompassing federated and continual learning University of Sydney
A distillation-based approach integrating continual learning and federated learning for pervasive services Inria
FedSpeech: Federated Text-to-Speech with Continual Learning Zhejiang University

 

19. Computational Learning Theory

Privacy, utility, and efficiency are the three key concepts of trustworthy federated learning. We point out that there is no security mechanism that can achieve optimality in terms of privacy leakage, utility loss, and efficiency loss simultaneously.

 

Google FL Workshops

 

Videos and Lectures

 

Tutorials and Blogs

 

Open-Sources

Developing a federated learning framework from scratch is very time-consuming, especially in industrial. An excellent FL framework can facilitate engineers and researchers to train, research and deploy the FL model in practice. In this section, we summarize some commonly used open-source FL frameworks from both industrial and academia perspectives.

Enterprise Grade

Platform Papers Affiliations/HomePage
FATE FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection WeBank
FedML FedML: A Research Library and Benchmark for Federated Machine Learning fedml.ai
OpenFL OpenFL: An open-source framework for Federated Learning Intel
NVFlare NVIDIA
IBM Federated Learning IBM Federated Learning: an Enterprise Framework White Paper IBM
Fedlearner Bytedance
PaddleFL Baidu
Sherpa.ai Federated Learning Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy Sherpa.ai
FederatedScope FederatedScope: A Flexible Federated Learning Platform for Heterogeneity Alibaba
secretflow Antgroup

 

Research Purpose

Platform Papers Affiliations/HomePage
Tensorflow-Federated Towards Federated Learning at Scale: System Design Google
FedJAX FEDJAX: Federated learning simulation with JAX Google
Flower Flower: A Friendly Federated Learning Research Framework flower.dev
FLUTE FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations Microsoft
FLSim Meta
PySyft A generic framework for privacy preserving deep learning OpenMined
PyVertical PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN OpenMined
LEAF LEAF: A Benchmark for Federated Settings CMU
FedScale FedScale: Benchmarking Model and System Performance of Federated Learning at Scale SymbioticLab(University of Michigan)
EasyFL EasyFL: A Low-code Federated Learning Platform For Dummies NTU
FedLab FedLab: A Flexible Federated Learning Framework SMILELab-FL
Galaxy Federated Learning GFL: A Decentralized Federated Learning Framework Based On Blockchain Zhejiang University
FedTree National University of Singapore

⬆ Return to top

About

Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond