Wide Neural Network(MNIST Digit Classification):
This is a prototype implementation of a Wide Neural Network. Details on the architecture will be added soon.
The primary architectural difference is that connections from every lower layer go to every higher layer.
Further, the final prediction is made only after neurons in all layers have fired. This set up makes gradient calculations and
backpropagation more complex.
The network is trained on the MNIST digit classification task.
The modules are written in Python and only use Numpy.
Completed:
Network initialization
Network Forward
Evaluate gradients - weights, v_in, and v_board
Softmax
Softmax Loss and derivative
Vectorization
Complete gradient implementation for biases
Update parameters with gradient
Numerical gradient checker
Test network gradients
To Do:
Train on mnist digit classification (in progress)