Juniorlimaivd / Neural-Networks-in-Keras

Neural-Networks-in-Keras

Keras is a python library for neural networks and provides a high-level inteface to Theano or Tensorflow libraries. The examples in this github repository assume that you are familiar with the theory of the neural networks. To learn more about the neural networks, you can refer the resources below.

Installation Instructions

Installation instructions for Keras (includes links to Theano and Tensorflow installaion): https://keras.io/#installation

To Use GPU, install CUDA: https://developer.nvidia.com/cuda-downloads

To resolve common installation issues, refer the README on Laura Graesser's Github repository.

Using Tensorflow as backend

Keras uses Tensorflow by default. Check the contents of ~/.keras/keras.json:

{
    "image_dim_ordering": "tf",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

Using Theano as backend

Edit the contents of ~/.keras/keras.json:

{
    "image_dim_ordering": "th", 
    "epsilon": 1e-07, 
    "floatx": "float32", 
    "backend": "theano"
}

Check the Backend being used

from keras import backend as K
print K.backend()
print K.image_dim_ordering()
## Set image dimesnion order for Tensorflow, so that no modifications are required in the code
if K.backend() =='tensorflow':
    K.set_image_dim_ordering("th")
    print K.image_dim_ordering()

Using GPU

For tensorflow, GPU is used by default. To use GPU for Theano, add this code in the beggining of the file:

import os
os.environ["THEANO_FLAGS"] = "mode=FAST_RUN, device=gpu, floatX=float32"

Resources:

Books:

• Neural Networks and Deep Learning by Michael Nielsen
• Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville

Video Lectures:

• Neural Networks for Machine Learning by Geoffrey Hinton
• Stanford Course CS231n: Video Lectures (Winter 2016)
• Udacity's Deep Learning Course
• Hugo Larochelle's lectures
• Videos from Bay Area Deep Learning School: Day1, Day2
• Machine Learning course by Nando de Freitas

Blogs:

• Article on Deep Learning by Yann LeCun, Yoshua Bengio and Geoffrey Hinton
• Christopher Olah's blog
• Stanford Course CS231n: Course webpage by Andrej Karpathy
• Overview of Gradient Descent Optimization Algorithms by Sebastian Ruder

Acknowledgements

Thank you to Laura Graesser for her Neural Networks workshop and WiMDS for organizing it.