MNIST Neural Network from Scratch

Objective: Build a neural network from scratch for the MNIST database.

Built from scratch with numpy
Optimized neural network with 98% precision after 20 epochs of training
Major changes from v1:

Switched from batch gradient descent to mini-batch gradient descent
Added momentum to descent algo
Better initialization of weights

Sample digits from the MNIST database (left) and a simple visual of how the neural network trains on them (right).

(Ali, 2024) contains python code for both v1 and v2 of the neural network.

Helpful Resources

Codecademy's courses on deep learning and neural networks
Official page for MNIST database (LeCunn et al., 1994)
Johnathan Weisberg's blogs on neural networks (Weisberg, 2018)

References

2024

mnistAli

MNIST Network From Scratch - Github Repository

Yusuf Ali

2024

Bib

2018

mnistWeisberg

Building a Neural Network from Scratch

Johnathan Weisberg

2018

Bib

@article{mnistWeisberg,
  title = {{Building a Neural Network from Scratch}},
  author = {Weisberg, Johnathan},
  year = {2018},
  url = {https://jonathanweisberg.org/post/A%20Neural%20Network%20from%20Scratch%20-%20Part%201/}
}

1994

mnistData
The MNIST Database of handwritten digits

Y. LeCunn, C. Cortes, and C. J.C. Burges

1994

Abs Bib

The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.
@article{mnistData, title = {The MNIST Database of handwritten digits}, author = {LeCunn, Y. and Cortes, C. and J.C. Burges, C.}, year = {1994}, url = {https://yann.lecun.com/exdb/mnist/}, }