tfjgeorge / nngeometry

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

Home Page:https://nngeometry.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

An example for gram matrix computation?

liutianlin0121 opened this issue · comments

Hi,

Thanks for this great resource! The existing example available is about FIM computation. I am wondering if you could also provide an example for computing gram matrix of empirical NTK of a finite-width network (perhaps with a dummy dataloader)? This will be highly helpful.

Since this is a request, please feel free to close this issue ;-)

Cheers,
Tianlin

Hi, thanks for your interest in NNGeometry!

I just added a quick example of a Gram matrix computation for the CIFAR10 10-class classification task on a ResNet18 model on the following ipynb notebook, or directly as a colab notebook.

Thanks a lot!