- Pytorch
- Matplotlib
- activation.py: contains classes of activation functions implemented
- layer.py: contains classes defining the layers
- loss.py: contains classes defining the losses
- nnmodule.py: parent class
- optimizer.py: contains classes defining optimizers
- parameter.py: contains the class Parameter
- sequential.py: contain the class Sequential
- test.py: contains an example of the test of this library
Only the linear fully connected layer has been implemented. The linear layer can be created in the following way: Linear(nb_input, nb_output).
ReLU, TanH and Sigmoid have been implemented. The activation function can be created in the following way: ReLU(), Tanh(), Sigmoid().
- Mean Squared Error: MSELoss()
- Binary Cross Entropy: BCELoss()
Loss classes have two functions:
- forward(output, target) which computes the loss of the model.
- backward(output, target) which computes the gradient of the loss with respect to the output of the model.
To create a model Sequential can be used in the following way: Sequential(layer1, activation1, ..., layern, activationn). Three functions are available from that class:
- forward(input) which computes the output of the model given the input.
- backward(gradwrtoutput) which triggers the backpropagation algorithm through all the layers of the model.
Only the SGD optimizer has been implemented. It can be created in the following way: SGD(model.parameters()).
- zero_grad() set to zero the gradient of the parameters of the models
- step() function updates the paramaters according to the gradient holded in the Parameter object.
To run the program how it is intended you need to execute test.py. You will first be asked to choose between two models. Just make your choice by inputing a 0 or 1 and pressing ENTER. You can also just input the ENTER key every time you're asked a choice to choose the default value. You will be then be asked how many epochs you want to train over. In the end, it will also ask you if you want to plot the results.