This is source code for paper Towards Addressing Label Skews in One-Shot Federated Learning.
An example running script of FedOV is in run.sh.
| Parameter | Description |
|---|---|
model |
The model architecture. Options: simple-cnn, vgg, resnet, mlp. Default = mlp. |
dataset |
Dataset to use. Options: mnist, cifar10, fmnist, svhn |
alg |
The training algorithm. Options: vote. |
lr |
Learning rate for the local models, default = 0.01. |
batch-size |
Batch size, default = 64. |
epochs |
Number of local training epochs, default = 5. |
n_parties |
Number of parties, default = 2. |
comm_round |
Should be 1 for one-shot FL. |
partition |
The partition way. Options: homo, noniid-labeldir, noniid-#label1 (or 2, 3, ..., which means the fixed number of labels each party owns) |
beta |
The concentration parameter of the Dirichlet distribution for heterogeneous partition, default = 0.5. |
device |
Specify the device to run the program, default = cuda:0. |
datadir |
The path of the dataset, default = ./data/. |
logdir |
The path to store the logs, default = ./logs/. |
init_seed |
The initial seed, default = 0. |
Some repos we refer to
General framework: https://github.com/Xtra-Computing/NIID-Bench
Adversarial attacks: directly use https://github.com/utkuozbulak/pytorch-cnn-adversarial-attacks
Cutpaste: use an unofficial reproduction from https://github.com/Runinho/pytorch-cutpaste. The authors of Cutpaste do not make code public.
Also include code from https://github.com/lwneal/counterfactual-open-set which is the code by authors of Open Set Learning with Counterfactal Images, ECCV 2018. However, since our FL partition settings are more diverse and complicated, we find it very hard to tune the hyper-parameters to generate good counterfactual images and the voting accuracy is low, so we do not call it in the final version.
In our code, we keep the commented or unused codes (functions). We tried these but did not get good results. After many trails and errors, we summarize the current DD and AOE functions. These trials may save efforts or bring some insights for future researchers, so we keep them.
For ResNet-50 experiments, since ResNet-50 has batch normalization layers, we have to mix train data and generated outliers in a batch, otherwise the model will become very bad. Please see folder resnet_code. Codes are like the following
x_con = torch.cat([x,x_gen11],dim=0)
y_con = torch.cat([target,y_gen],dim=0)
loss = criterion(out, y_con)
If you find our work useful, please cite
@inproceedings{
diao2023towards,
title={Towards Addressing Label Skews in One-Shot Federated Learning},
author={Yiqun Diao and Qinbin Li and Bingsheng He},
booktitle={International Conference on Learning Representations},
year={2023},
url={https://openreview.net/forum?id=rzrqh85f4Sc}
}