Introduction to Artificial Neural Network


This and  upcoming post we will cover Artificial neural network and its types ,understanding these topics requires some mathematical understanding of calculus and linear algebra.

Goal : To make understand how a single neuron in an artificial neural network work .

What is Artificial neuron ?

There is not a single Definition exists that can define an artificial neuron some scientist in the design of a artificial neuron use biological neuron as a reference model ,some uses math’s as the base to define it,But the simplest form of a artificial neuron is Perceptron model which looks like this,

Artificial neuron
Biological Neuron

Here all w’s are weights multiplied with all input x’s  ,this is primarily done to define importance  of some inputs over other and then this is passed through an activation function and based upon the output of that activation function the neuron gets activated or not.All this functioning seems very similar to an functioning of a biological neuron but in reality a biological neuron is much more complex.

So ,lets understand how this perceptron model works:


  • Initialize the weights and the bias. Weights may be initialized to 0 or to a small random value.Random initialization is preferred
  • For Each example i to n in our training set X do following till error is minimized
    • calculate the output y
      • Y = W_{{}}\ast X_{{i}} +W_{{0}}
      • pass Y in your choice of activation function in our case let’s take sigmoid activation function
        • A = f_{{x}}(W_{{}}\ast X_{{i}} +W_{{0}})
      • Calculate the loss by using a loss function in our case it will be
        • L=max(0,−yi∗Ai), where y is taget label and A is predicted label.
      • Update the weights
        • W_{{i}}(t+1) = W_{{i}} +\alpha\ast derivative- w.r.t (W_{{i's}})(Loss-function)

Following approach is very similar to logistic regression as perceptron model is highly inspired by logistic regression.In following coming article we will cover how to link these single neuron model to built a Artificial neural network so stay tuned.

About the author


I write blogs about Machine Learning and data science

By abhinavsinghml

Most common tags

%d bloggers like this: