shreyansh26 / Experiments-with-the-Neural-Tangent-Kernel

Learning more about the NTK through existing paper implementations

Experiments with Neural Tangent Kernel (NTK)

These are some paper implementations and other code snippets to help me learn more about Neural Tangent Kernels. Most of this is from existing implementations and I am maintaining this just for learning purposes.

The notebooks currently in the repository are for -

• Eignedecomposition of the empirical NTK
• Training Neural Networks (non-linear) and Linear models (after linearization) on binary eigenfunctions of the NTK at initialization
• The training and loss landscape of the linear models at initialization vs neural networks vs linear model with kernel extracted after non-linear pretraining

References

• Neural Tangents library by Google
• Source code of "What can linearized neural networks actually say about generalization?"