blaisedelattre / lip4conv

Code for Spectral Norm of Convolutional Layers with Circular and Zero Paddings and Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration

Spectral Norms and Lipschitz Bounds for Convolutional Layers

This repository contains the code for the following articles:

• Spectral Norm of Convolutional Layers with Circular and Zero Paddings by Blaise Delattre, Quentin Barthélemy, Alexandre Allauzen
• Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration published at ICML 2023 by Blaise Delattre, Quentin Barthélemy, Alexandre Araujo and Alexandre Allauzen.

Gram iteration is a deterministic method to compute spectral norm in quadratic convergence. It exhibits SOTA results on GPU regarding spectral norm computations.

This repository

Outline

• bounds.py contains code for different spectral norm bounds.

• note_book_test_gram_iteration.ipynb contains some examples of spectral norm bound computations for different methods on dense and convolutional layers.

• train_local.py contains code to launch a training. Start a default configuration run python train_local.py --bound delattre2023 --bound_n_iter 6 --lr 0.1 --r 0.1

Installation

Experiences were done using pytorch-cuda=11.7

git clone https://github.com/blaisedelattre/lip4conv.git