How to generate the figure 4 in your paper?

[DoranLyong]

I've been really waiting for you to upload the full code.
Anyway, congratulation on your work accepted in NeurIPS2022~

Question1

I'm sort of confused about this figure.
Is it spectrum maps of features on each branch?
Or spectrums of weight kernels?

Question2

Could you share some code to generate this figure?
I've tried but I could not like this clean figure.

[yongxin]

I have also tried to implement this functionality. However，Encountering the same difficulty, the size of the feature map makes the transformed Fourier spectrum resolution very low. Looking forward to your answers and guidance!

[ChenyangSi]

def freq_fun(all_feat_layer, save_path):
B, C, H, W = all_feat_layer.shape
print(all_feat_layer.shape)
all_feat_layer = all_feat_layer.numpy()

freq_view_all = []
for i in range(B):

    feat = all_feat_layer[i]
    freq = np.fft.fft2(feat)
    freq = np.fft.fftshift(freq)
    freq_view = np.log(1 + np.abs(freq))

    freq_view_all.append(freq_view.mean(0))
freq_view_all = np.stack(freq_view_all, axis=0)
freq_view_all = freq_view_all.mean(0)
v_min = freq_view_all.min()
v_max = freq_view_all.max()
h, w = freq_view_all.shape
latent = np.diag(freq_view_all)[int(h/2):]  # only use the half-diagonal components
latent = latent - latent[0]  # visualize 'relative' log ampli

freq_view = (freq_view_all - v_min) / (v_max - v_min) * 255
freq_view = cv2.resize(freq_view, (112, 112))

# print(freq_view.shape)

freq_view = freq_view.astype(np.uint8).copy()
freq_view = cv2.applyColorMap(freq_view, cv2.COLORMAP_JET)
cv2.imwrite(save_path+'freq_img.png', freq_view)

[ChenyangSi]

@DoranLyong
yes, it is the spectrum of feature maps on each branch.