Artificial Neural Network for Dummies

What is a Neural Network ?

The term Neural Network in Machine Learning goes in analogy with the biological Neural Network of a Human brain! The human brain is made up of about 100 billion neurons. These neurons make the core of our brain and our nervous system. Every neuron is made up of the nucleus (the core), the axon (the filament) and dendrites (the connectors). The interface through which neurons interact with their neighbors usually consists of several axon terminals connected via synapses to dendrites on other neurons.

When sensory organs generate impulses (say when you touch a table), they are transmitted along a network of interconnected neurons, which process the impulse one after the other until they trigger a physiological response. And so the final physiological response to the nerve impulse is a culmination of the entire intermediary processing in the network.

Artificial Neural Network is a Supervised Machine Learning algorithm which has somewhat a similar structure and functioning as the neurons in the brain.

Underlying structure of a Neural Network

So to give you an idea, lets talk about a very basic Neural Net. It consists of the following components:

• Neurons: Just like their biological counterparts, they are the elementary units in an artificial neural network. The artificial neuron receives one or more inputs and sums them to produce an output.
• Connections and weights: They are the connections between the neurons. These connections are provided with some weights/parameters. When ever an input is received by the neurons, they weights/parameters changes in these connections before the output is given to a different neuron to process. The weights can be positive or negative. We will talk more about later.

• Propagation: The propagation algorithm is the workhorse of learning in neural networks. Commonly used back-propagation is used to adjust the above mentioned weights in between the connections. After each neuron processes a batch of data, back-propagation algorithm calculates the error contribution of each neuron and adjusts the weights of the neuron. The algorithm generally uses Loss Function, that calculates the difference between the network output vs its expected output after an input propagates through the network. The resulting error value (difference) is then calculated for each of the neurons in the output layer. The error values are then propagated from the output back through the network, until each neuron has an associated error value that reflects its contribution to the original output. Back-propagation uses these error values to calculate the gradient of the loss function which is then fed to the optimization method, which in turn uses it to update the weights, in an attempt to minimize the loss function.
• Learning Rule: This is an iterative process which helps neural network to improve its performance and learn from existing conditions. There are many types of rules such as Hebbian learning rule, Perceptron learning rule, Delta learning rule, Correlation learning rule, Outstar learning rule etc. These rules basically gives neural networks an ability to learn and improve, a basic prerequisite of a machine learning algorithm.

How does it work ?

Now coming to the main point: How does it work? Well with its structure being explained, its very easy to now explain working of a very basic Neural Network.

Consider a simple example. Think of a Guitar!! Neural Network is like a collection of strings in a guitar. In order to use it to play a song, you need to tune it up. What is the process of tuning? You send in an input to a string of the guitar to hear an output. If the output doesn't sound like to what it should, you tighten or loosen the string. This require the same process with other strings to bring it to a point where you get a desired pitch. In the same way, each string is a network in a Neural Network and the process with which you fine tune a string in the guitar is called Back-Propagation in Neural Net. How does the tuning happen ? The network changes the weights associated with each connection between the neurons.

So in the same way, the input neurons receive an input signal, which it then multiplies with the weight assigned to each connection and then passes it to the next neuron. Every unit adds up all the inputs it receives in this way and if the sum is more than a certain threshold value, the unit "fires" and triggers the units it's connected to (those on its right). At its output, it compares the output a network produces with the output it was meant to produce, and using the difference between them to modify the weights of the connections between the units in the network, working from the output units through the hidden units to the input units—going backward, technique known as Back-Propagation.

In time, back-propagation causes the network to learn, reducing the difference between actual and intended output to the point where the two exactly coincide (which also leads to over fitting most of the times, which needs to be taken care of!), so the network figures things out exactly as it should. This process is used during the training of the data. Once the network has been trained with enough train sample, the model is then used on the test data to see how accurately it predicts. This is the working of a very basic neural net in a very very simplified manner. Though the ones which we use in real life are much more complex, but they all pretty much follow the same principle.

Conclusion:

When I was starting my career in data science and machine learning, the big words such as Neural Network used to always haunt me till the time I implemented one of these. I realized that its not difficult to understand provided you have properly done your research and learned its concept from logical stand point. I will try to come up with another article about the various types of Neural Nets being used in the industry today and also try to explain them from math and statistics standpoint this time. I tried to give reasonable explanation on this topic, but if you feel I miss-represented something, please leave a comment on it so that I can improve this article. Feel free to like and share if you like this article. Thank you for your time.