Need help with mlp-classifier?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

206 Stars 49 Forks MIT License 13 Commits 8 Opened issues


A handwritten multilayer perceptron classifer using numpy.

Services available


Need anything else?

Contributors list

# 301,415
13 commits

MLP Classifier

A Handwritten Multilayer Perceptron Classifier

This python implementation is an extension of artifical neural network discussed in Python Machine Learning and Neural networks and Deep learning by extending the ANN to deep neural network & including softmax layers, along with log-likelihood loss function and L1 and L2 regularization techniques.

Some Basics

An artificial neuron is mathematical function conceived as a model of biological neurons. Each of the nodes in the diagram is a a neuron, which transfer their information to the next layer through transfer function.


The transfer function is a linear combination of the input neurons and a fixed value - bias (threshold in figure). The coefficients of the input neurons are weights.
In the code, bias is a numpy array of size(layers-1) as input layer do not have a bias. The weights, also a numpy array, form a matrix for every two layers in the network.
Activation function is the output of the given neuron.

X: vectorize{(j-1)th layer}

w = weights[j-1]
bias = threshold[j-1]
transfer_function = dot_product(w, X)
o = activation(transfer_function + bias)


The implementation includes two types of artificial neurons: * Sigmoid Neurons
* Softmax Neurons

The loss function associated with Softmax function is the log-likelihood function, while the loss function for Sigmoid function is the the cross-entropy function. The calculus for both loss functions have been discussed within the code.

Further, the two most common regularization techiques - L1 and L2 have been used to prevent overfitting of training data.

Why Softmax?

For zLj in some vector ZL, softmax(zLj) is defined as

The output from the softmax layer can be thought of as a probability distribution.

In many problems it is convenient to be able to interpret the output activation O(j) as the network's estimate of the probability that the correct output is j.

Refer these notes for calculus of softmax function.
Source of MNIST training data-set.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.