Excuse me, I wonder which layer have you been used to run PCA?

[DemoAllan]

Excuse me, sir. When I was reading the paper of RFA, I tried to figure out which layer's weights were you used to run PCA, and how did you do it?
Thank you for answering my question.

[Krishna Pillutla]

Hello,

The plots stack all the weights of the network into a single vector (of size n_dim in the code below). For the linear model, this vector is obtained by stacking the weights and the biases of the only linear layer. Here is some sample code. Hope this answers your question.

# w contains all vectorized updates of all participating stacked into a matrix; w.shape = (n_devices, n_dim) 
# corrupted_idxs is an array containing indices of corrupted devices
# clean_idxs is an array containing indices of un-corrupted devices
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
pca = PCA(n_components=2, random_state=0)
proj_w = pca.fit_transform(w)
plt.scatter(proj_w[clean_idxs, 0], proj_w[clean_idxs, 1], label='Not corrupted')
plt.scatter(proj_w[corrupted_idxs, 0], proj_w[corrupted_idxs, 1], label='Corrupted')