The goal of this course was to:
- Develop solutions based on Neural Networks using PyTorch
- Acquire a deep knowledge on deep learning and differente network architectures
- Solve tasks using MLP's
- Solve tasks using Convolutional Neural Networks (CNN's)
- Solve tasks using Recurrent Neural Networks (RNN's)
- Acquire knowledge about Transformers, Variational Auto Encoders (VAE's), Generative Adversarial Networks (GAN's) and Diffusion Models
This course consisted in four projects, two partial tests, a Kaggle competition and a final test.
In this first project, we were asked to construct an Autoencoder and train it on the MNIST and FMNIST dataset. For it we would have to explore differente architectures and apply regularization techniques and study its effects.
This second project was based on the following paper: On Calibration of Modern Neural Networks. This paper describes the problem of calibration that exists in modern neural networks, such as overconfidence or under confidence.
During this project we had to use a LeNet-5 network and train on a modified version of CIFAR-10 dataset. We were asked to study the calibration curve and implement one of the solutions presented in the paper, specifically the one called Temperature Scalin, and study its effects.
Then there was an optional part where we were asked to repeat the experiment with a larger network.
This third project consisted of implenmenting the attention mechanism to Recurrent Neural Network with LSTM. We were asked to code the attention module from scratch, since it had to be coded following certain instructions presented at the beginning of the notebook. This was an excellent practical approach to implement the attention mechanism from scratch due to its importance.
Finally we are asked to compare the implemented solution with the regular RNN with LSTM without attention.