tfjgeorge / nngeometry

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

Home Page:https://nngeometry.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

compute_correlation() intent

perone opened this issue · comments

Hi @tfjgeorge, thanks for the library, a very useful framework. I have a quick question regarding the compute_correlation you're calling in the notebook, why is that being done before visualization of the FIM ? Not sure if I understood what is the intention. Thank you !

Hi, thank you for your interest in NNGeometry.

compute_correlation takes a covariance matrix, and computes the corresponding correlation matrix (i.e. by normalizing by the standard deviation). It is just more visually appealing to plot the correlation rather than looking at a covariance matrix, because otherwise we would just be looking at a matrix with very few high components (yellow pixels in the notebook), so basically a dark blue square with very few yellow pixels.

I suggest you just try removing compute_correlation to just see the difference.

Best, Thomas

Thanks @tfjgeorge, make total sense, I tried also to visualize it but was struggling w/ the colormaps, so I suspected it would be because of viz. Thanks again for the open-sourcing it!