rharish101 / DLGeneralization

My project to recreate the results in "Understanding Deep Learning Requires Rethinking Generalization"

Deep Learning Generalization

My project to recreate the results in the paper "Understanding Deep Learning Requires Rethinking Generalization" and do a similar analysis on LSTMs in NLP.

Contents

• Code

  • Mini Alexnet

    • cifar10_alexnet.py: Code to train mini Alexnet on CIFAR10 without regularization.
    • cifar10_alexnet_rand_crop.py: Code to train mini Alexnet on CIFAR10 without regularization and with random crop.
    • cifar10_alexnet_rand_labels.py: Code to train mini Alexnet on CIFAR10 without regularization and with random training labels.
    • cifar10_alexnet_wd.py: Code to train mini Alexnet on CIFAR10 with weight decay.
    • cifar10_alexnet_wd_rand_crop.py: Code to train mini Alexnet on CIFAR10 with weight decay and with random crop.

  • Mini Inception

    • cifar10_tf_inception.py: Code to train mini Inception on CIFAR10 without regularization.
    • cifar10_tf_inception_rand_crop.py: Code to train mini Inception on CIFAR10 without regularization and with random crop.
    • cifar10_tf_inception_wd.py: Code to train mini Inception on CIFAR10 with weight decay.
    • cifar10_tf_inception_wd_rand_crop.py: Code to train mini Inception on CIFAR10 with weight decay and with random crop.

  • MLP1 - MLP with 1 hidden layer

    • cifar10_keras_mlp.py: Code to train MLP with 1 hidden layer on CIFAR10 without regularization.
    • cifar10_keras_mlp_rand.py: Code to train MLP with 1 hidden layer on CIFAR10 without regularization and with random training labels.
    • cifar10_keras_mlp_wd.py: Code to train MLP with 1 hidden layer on CIFAR10 with weight decay.

  • MLP3 - MLP with 3 hidden layers

    • cifar10_keras_mlp3.py: Code to train MLP with 3 hidden layers on CIFAR10 without regularization.
    • cifar10_keras_mlp3_rand.py: Code to train MLP with 3 hidden layers on CIFAR10 without regularization and with random training labels.
    • cifar10_keras_mlp3_wd.py: Code to train MLP with 3 hidden layers on CIFAR10 with weight decay.

• Saved Models

  • Mini Alexnet

    • cifar10_alexnet.h5: A saved Keras Sequential model for mini Alexnet trained on CIFAR10.
    • cifar10_alexnet_rand_crop.h5: A saved Keras Sequential model for mini Alexnet trained on CIFAR10 with random crop.
    • cifar10_alexnet_rand_labels.h5: A saved Keras Sequential model for mini Alexnet trained on CIFAR10 with random labels.
    • cifar10_alexnet_wd.h5: A saved Keras Sequential model for mini Alexnet with L2 regularization, trained on CIFAR10.
    • cifar10_alexnet_wd_rand_crop.h5: A saved Keras Sequential model for mini Alexnet with L2 regularization, trained on CIFAR10 with random crop.

  • MLP1 - MLP with 1 hidden layer

    • cifar10_keras_mlp.h5: A saved Keras Sequential model for MLP with 1 hidden layer, trained on CIFAR10.
    • cifar10_keras_mlp_rand.h5: A saved Keras Sequential model for MLP with 1 hidden layer, trained on CIFAR10 with random labels.
    • cifar10_keras_mlp_wd.h5: A saved Keras Sequential model for MLP with 1 hidden layer and L2 regularization, trained on CIFAR10.

  • MLP3 - MLP with 3 hidden layers

    • cifar10_keras_mlp3.h5: A saved Keras Sequential model for MLP with 3 hidden layers, trained on CIFAR10.
    • cifar10_keras_mlp3_rand.h5: A saved Keras Sequential model for MLP with 3 hidden layers, trained on CIFAR10 with random labels.
    • cifar10_keras_mlp3_wd.h5: A saved Keras Sequential model for MLP with 3 hidden layers and L2 regularization, trained on CIFAR10.

• Results

  • Mini Alexnet

    • Alexnet_Cifar10.png: Mini Alexnet with no regularization.
    • Alexnet_Cifar10_rand_crop.png: Mini Alexnet with no regularization and with random crop.
    • Alexnet_Cifar10_rand_labels.png: Mini Alexnet with no regularization and with random labels.
    • Alexnet_Cifar10_wd.png: Mini Alexnet with weight decay.
    • Alexnet_Cifar10_wd_rand_crop.png: Mini Alexnet with weight decay and with random crop.

  • Mini Inception

    • Inception_Cifar10.png: Mini Inception with no regularization.
    • Inception_Cifar10_rand_crop.png: Mini Inception with no regularization and with random crop.
    • Inception_Cifar10_wd.png: Mini Inception with weight decay.
    • Inception_Cifar10_wd_rand_crop.png: Mini Inception with weight decay and with random crop.

  • MLP1 - MLP with 1 hidden layer

    • MLP1_Cifar10.png: MLP with 1 hidden layer with no regularization.
    • MLP1_Cifar10_rand_labels.png: MLP with 1 hidden layer with no regularization and with random labels.
    • MLP1_Cifar10_wd.png: MLP with 1 hidden layer with weight decay.

  • MLP3 - MLP with 3 hidden layers

    • MLP3_Cifar10.png: MLP with 3 hidden layers with no regularization.
    • MLP3_Cifar10_rand_labels.png: MLP with 3 hidden layers with no regularization and with random labels.
    • MLP3_Cifar10_wd.png: MLP with 3 hidden layers with weight decay.