We provide a friendly foundational platform for beginners who intend to start Federated Learning (FL).[1]
Federated learning (FL), proposed by Google at the very beginning, is recently a burgeoning research area of machine learning, which aims to protect individual data privacy in the distributed machine learning processes, especially in finance, smart healthcare, and edge computing. Different from traditional data-centered distributed machine learning, participants in the FL setting utilize localized data to train local models, then leverages specific strategies with other participants to acquire the final model collaboratively, avoiding direct data-sharing behavior.
To ease the hassle of implementing the FL algorithm for beginners, we set up a highly customizable framework Fed-DP in this work. Fed-DP provides the necessary modules for FL, such as basic model, model optimization, data partitioning, etc. In the future, our framework will also include the following:
- Shuffle Model (SM) [2]
- Differential Privacy (DP) [3]
-
- Shuffle Model and Differential Privacy (SDP) [2,5]
- Personalized Federated Learning (PFL) [4]
Model | Paper | Time | Type | Official Code | Fed-DP Code | Methods | DP |
---|---|---|---|---|---|---|---|
FedAvg | Communication-Efficient Learning of Deep Networks from Decentralized Data | AISTATS 2017 | FL | None | Code | CDP,LDP | |
FedProx | Federated Optimization in Heterogeneous Networks | MLsys 2020 | PFL | Code | Code | Regularized Local Loss | CDP,LDP |
FedNova | Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization | NeurIPS 2020 | PFL | Code | Code | Regularized Local Loss | CDP,LDP |
SCAFFOLD | SCAFFOLD: Stochastic Controlled Averaging for Federated Learning | ICML 2020 | PFL | None | Code | Regularized Local Loss | CDP,LDP |
MOON | Model-Contrastive Federated Learning | CVPR 2021 | PFL | Code | Code | Regularized Local Loss | CDP,LDP |
FLAME | FLAME: Differentially Private Federated Learning in the Shuffle Model | AAAI 2021 | LDP-FL | Code | Code | SDP | CDP,LDP,SDP |
Command to create a new environment in anaconda
git clone https://github.com/NigeloYang/Fed-DP.git
cd Fed-DP
conda env create -f Fed_dp_linux.yaml/Fed_dp_win.yaml
conda activate env_name
This project has generated Ten data distributions of MNIST, Fashion-Mnist and Cifar-10, as follows:
- Balanced-IID (two)
- Unbalanced-IID (two)
- Balanced Shards
- Unbalanced Shards
- Quantity-Based-Label-Distribution
- Hetero-Dirichlet
- Balanced-Dirichlet
- Unbalanced-Dirichlet
Generation Data Methods:
cd Fed-DP/dataset
python generate_data.py # for MNIST iid_balanced_100
For details, please see: Generate Datasets (updating)
-
Build dataset: Datasets (updating)
-
Train and evaluate the model:
python main.py --algorithm=FedAvg # for FedAvg and MNIST,dataset distribution --dataiid=1
For technical issues related to Fed-DP development, please contact me through Github issues or email: :satisfied:
- 📧 yangqiantao@126.com
- Base Federated Learning Framework
- Integrated Differential Privacy (Updating,next add meta`s DP tool -> opacus )
- Integrated shuffle Model (Updating)
- Integrated Personalized Federated Learning to solve Non-IID (Updating)
- Generated Data: IID, Non-IID (Updating)
[1] Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019). Federated Machine Learning: Concept and Applications. ACM Trans. Intell. Syst. Technol., 10(2), 12:11-12:19.
[2] Cheu, A. (2021). Differential Privacy in the Shuffle Model: A Survey of Separations. CoRR, abs/2107.11839.
[3] Cynthia, D., & Aaron, R. (2014). The Algorithmic Foundations of Differential Privacy.
[4] Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2022). Towards Personalized Federated Learning. IEEE Transactions on Neural Networks and Learning Systems, 1-17. Chinese
[5] Balle, B., et al. (2019). The Privacy Blanket of the Shuffle Model: 638-667.