Implementation of Does the Data Induce Capacity Control in Deep Learning? (ICML 2022)
Install backpack package https://github.com/f-dangel/backpack
pip3 install backpack-for-pytorch
Experiments calculating eigenvalues, eigenvectors, overlaps and analytical PAC-Bayes bounds: exp_all.ipynb
Experiments on random datasets: random sloppy.ipynb
Experiments calculating PAC-Bayes bounds:
Fully-connected net, reproduction of https://arxiv.org/abs/1703.11008
python bayes1.py --num_neurons 600 --num_layers 2
Fully-connected net, Method2:
python bayes_kfac1.py --num_neurons 600 --num_layers 2
Fully-connected net, Method3:
python bayes_kfac2.py --num_neurons 600 --num_layers 2
Fully-connected net, Method4:
python bayes_kfac.py --num_neurons 600 --num_layers 2
LENET, reproduction of https://arxiv.org/abs/1703.11008
python bayes2.py
LENET, Method2:
python bayes_proj.py --method method2
LENET, Method3:
python bayes_proj.py --method method3
LENET, Method4:
python bayes_proj_prior.py