This project will use Deep Learning technique to solve mnist digit hand writing problem.
For ease of writing code, I use keras, it's easy and intuitive.
For the DNN version, I borrowed it from keras example. After test, its performance is not good.
Then I tried the CNN version from this article(a little difference):
Multi-column Deep Neural Networks for Image Classification
The performance is exciting.
The next step I want to compare the CNN and LSTM on this dataset and learn more basic idea about it.
DNN: FC128-FC128-FC10-SoftMax
CNN:
30 epoch in 21000s ('Test score:', 0.20866020459792831)
CNN2: CV4(55)-CV8(33)-MP22-CV16(33)-MP2*2-FC-128-FC10-SoftMax
10 epoch in 1150s ('Test score:', 0.070545128054809794)
CNN2 with relu activation:
10 epoch in 1000s ('Test score:', 2.3021199735210596)
CNN2 with dropout 0.5 before flatten
10 epoch in 960s ('Test score:', 2.3021494342803526)
CNN3: CV4(55)-CV8(33)-MP22-CV16(33)-CV32(22)-MP22-FC128-FC10-Softmax
10 epoch in 2400s
loss: 0.0266 - acc.: 0.9912 - val. loss: 0.0523 - val. acc.: 0.9848
('Test score:', 0.049439467147584119)
CNN4: CV8(55)-CV16(33)-MP22-CV32(33)-CV64(22)-MP22-FC256-FC10-Softmax
loss: 0.0045 - acc.: 0.9987 - val. loss: 0.0486 - val. acc.: 0.9882
python mnist.py 6665.99s user 87.56s system 101% cpu 1:51:20.24 total
('Test score:', 0.03452320910848105)