Need help with Neural-Network-from-scratch?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

243 Stars 59 Forks 27 Commits 1 Opened issues


Ever wondered how to code your Neural Network using NumPy, with no frameworks involved?

Services available


Need anything else?

Contributors list

Learn backpropagtion the hard way


In this repository, I will show you how to build a neural network from scratch (yes, by using plain python code with no framework involved) that trains by mini-batches using gradient descent. Check for the code.

In the related notebook NeuralNetworkfromscratchwith_Numpy.ipynb we will test on a set of non-linear classification problems

  • We'll train the neural network for some number of epochs and some hyperparameters
  • Plot a live/interactive decision boundary
  • Plot the train and validation metrics such as the loss and the accuracies

Example: Noisy Moons (Check the notebook for other kinds of problems)

Decision boundary (you'll get to this graph animated during training)

Decision boundary

Loss and accuracy monitoring on train and validation sets

Loss/Accuracy monitoring on train/val

Where to go from here? is a toy neural network that is meant for educational purposes only. So there's room for a lot of improvement if you want to pimp it. Here are some guidelines:

  • Implement a different loss function such as the Binary Cross Entropy loss. For a classification problem, this loss works better than a Mean Square Error.
  • Make the code generic regarding the activation functions so that we can choose any function we want: ReLU, Sigmoid, Tanh, etc.
  • Try to code another optimizers: SGD is good but it has some limitations: sometimes it can be stuck in local minima. Look into Adam or RMSProp.
  • Play with the hyperparameters and check the validation metrics

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.