This is a Digit Recognizer Neural Network using only Pandas and NumPy on the MNIST Dataset
The goal of this project was to gain a deeper understanding of neural networks and their inner workings. Initially, the complexity of concepts like backpropagation, activation functions, and gradient descent seemed overwhelming. To overcome this, I decided to implement a neural network from scratch without using any libraries or wrappers. This hands-on approach allowed me to grasp the foundational concepts and gain a solid understanding of the underlying principles.
Fortunately, there were others who had undertaken similar projects, and I leveraged their experiences and resources available on the internet to build my own version.
Throughout this project, I gained knowledge in several key areas, including:
- Backpropagation: I learned how to propagate errors backward through the network, updating weights and biases to improve the network's performance.
- Activation Functions: I explored different activation functions and their significance in introducing non-linearity to the network, enabling it to learn complex relationships.
- Gradient Descent Techniques: I studied various techniques in gradient descent, such as batch gradient descent, to optimize the network's learning process.
- Setting a Good Learning Rate (Alpha): I discovered how to choose an appropriate alpha value for the gradient descent function, ensuring efficient convergence.
These were just a few of the many small but important details I encountered during this project, which greatly enhanced my understanding of neural networks.
The entire project took approximately 2-3 days. During this time, I devoted myself to learning the necessary concepts and implementing the neural network from scratch. The process was enjoyable and rewarding, as it allowed me to gain hands-on experience and deepen my knowledge.
I would like to share the resources that proved invaluable in my journey of understanding neural networks:
- But what is a neural network? | Chapter 1, Deep learning - YouTube
- Building a neural network FROM SCRATCH (no Tensorflow/Pytorch, just numpy & math) - YouTube
- Neural Networks Explained from Scratch using Python - YouTube
- How to Create a Neural Network (and Train it to Identify Doodles) - YouTube
- Learning rate alpha in gradient descent | by Thamarasee Jeewandara | Medium
- Simple MNIST NN from scratch (numpy, no TF/Keras) | Kaggle
These resources provided valuable insights and guidance throughout my learning process, and I hope they prove helpful to you as well.