TL-System / plato

A federated learning framework to support scalable and reproducible research

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[BUG]The previous model weights are not loaded at the beginning of each round (for differential privacy).

tianzheng4 opened this issue · comments

Describe the bug
The previous model weights are not loaded at the beginning of each round. In each round, the model is trained on a randomly initialized model not the previous model. (for differential privacy).

It seems that the problem is not in the train_model function in diff_privacy.py. I print the weights of the first convolution layer before self.make_model_private() in train_model in diff_privacy.py for each round, the weights are always
conv1.weight tensor([[[[ 0.0702, 0.0594, 0.1692, -0.0529, -0.1473],
[ 0.0173, -0.1123, -0.0269, -0.1801, -0.0909],......

To Reproduce
Steps to reproduce the behavior:

  1. Enable differential privacy in fedavg_lenet5.yml
    image

  2. Set total_clients and per_round as 1

  3. Execute ./run -c ./configs/EMNIST/fedavg_lenet5.yml

Expected behavior
At the beginning of each round, the model should load the model weights at the end of the previous round.

Screenshots
At the end of the first round (the loss becomes smaller around 3.2)
image
At the beginning of the second round (the loss is back to 3.85, which is a very likely to be the result of a reinitialized model)
image

OS environment (please complete the following information):

  • OS: Ubuntu Linux