A small project that was inspired by Mark Kraay's Video, where he created a Neural Network from Scratch in C. This led me to a rabbit hole in machine learning, where I started learning about the mathematics behind machine learning, forward propagation, backward propagation, neural network architecture, and so on. Now, it would be boring if all I learned are theories, thus I wanted to try making my own rendition of a Neural Network in C, using the MNIST digit dataset.
- Install a C compiler, preferably GCC.
- [Optional] Install Make to make use of
MakeFileand instantly compile binaries.
Without MakeFile:
mkdir output
gcc lib/stats.c -o output/stats.o -c
gcc lib/matrix.c -o output/matrix.o -c
gcc lib/doubly_ll.c -o output/doubly_ll.o -c
gcc lib/image_set.c -o output/image_set.o -c
gcc lib/neural_net.c -o output/neural_net.o -c
gcc lib/ml.c -o output/ml.o -c
gcc main.c -o output/main.o -c
cd output
gcc -o ../mnist main.o stats.o matrix.o doubly_ll.o image_set.o neural_net.o ml.o
cd ..
rm -rf outputWith MakeFile:
makeYou can then run the compiled mnist.exe program using by typing in the console: ./mnist or make run if MakeFile is installed.
There are currently 6 libraries that I created for this project. They are completely reusable depending on the needs of your project. However, do take note of their header files and dependencies when copying. The documentation for the functions stored in these libraries can be found in their respective header files.
| Library | Dependencies | Description |
|---|---|---|
| stats | none | A utility library which contains different statistical functions. |
| matrix | none | A library for working with matrices. |
| doubly_ll | none | A library for working with doubly linked list. |
| image_set | matrix, ml | A library for working with the MNIST digit dataset. |
| neural_net | matrix, doubly_ll | A library for creating and working with neural networks. |
| ml | matrix, stats, neural_net | A library for training and testing neural networks against a dataset. |
- 3Blue1Brown - Deep Learning Series - Very intuitive look into neural networks and machine learning.
- Neural Networks/Deep Learning - This playlist is what made it click for me, especially in understanding back propagation.
- Gradient Descent
- Samson Zhong - Building a Neural Network From Scratch (Numpy & Maths)
- Neural Network from Scratch | Mathematics & Python Code
- Normalization VS Standardization of Data
- What is a Makefile and how does it work?
- Weight Initialization Techniques in Neural Networks
- Bias Initialization in a Neural Network
- Activation Functions