parampopat / Lottery_Ticket_Hypothesis-TensorFlow_2

Implementing "The Lottery Ticket Hypothesis" paper by "Jonathan Frankle, Michael Carbin"

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks - Using TensorFlow 2.0

A GitHub repository implementing The Lottery Ticket Hypothesis paper by Jonathan Frankle, Michael Carbin

"lottery ticket hypothesis:" dense, randomly-initialized, feed-forward and/or convolutional networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.

The paper can be downloaded from: The Lottery Ticket Hypothesis

Work done so far:

1. MNIST dataset using 300-100-10 Dense Fully connected neural network winning ticket identification.
2. MNIST dataset using LeNet-5 Convolutional Neural Networks.
3. Validation of the winning ticket identified for MNIST and CIFAR-10 dataset using relevant neural networks.
4. Conv-2/4/6 Convolutional Neural Network (CNN) for CIFAR10 dataset; pruning network till 0.5% of original connections remain and observe training and testing accuracies and losses.
5. Weight Pruning implementations using: numpy based unstructured pruning and tensorflow_model_optimization toolkit based pruning.

Prerequisites for the code to run:

• Python 3.X
• numpy 1.17 and/or above
• TensorFlow 2.0
• tensorflow_model_optimization