Rolling in the Deep Learning: Basic Concepts for Everyone

Simple learning adventure in under 8 minutes.

Alright, Alright, Alright! You probably know today deep learning is more relevant than ever. It is like synonymous with the modern era and lots of people love throwing this term around in conversation on social media. But why is it so? While the majority are just wowing, you can look deeper and find the answer.

Wanna catch a deep learning mechanism and finally know how everything works? If your answer is yes, then it is time to solve the mystery and know the magic behind it. In my attempts, I have done my best to help you with this matter and so I had showcased all important concepts regarding deep learning. 

If you are a newcomer, don’t worry, cause I tried to avoid complicated terms, code pieces or math parts. It’s totally an easy and comprehensive manual for everyone, so grab your coffee and a comfortable chair and enjoy your learning.

Part I: General Definitions

Deep Learning, Machine Learning, and Artificial Intelligence - all together these cutting-edge technologies are connected, but not the same. For every curious beginner, it is better to start learning adventure with clarifying the differences between these buzzwords and see the whole picture. 

In short, this picture has every element that comes out one after another. AI is an umbrella that contains everything: AI absorbs ML, and ML, in turn, absorbs DL. Here is a brief overview to figure it out better:

Artificial intelligence is a broad concept. At its core, AI is the presence of a wide variety of algorithms designed to simulate a human solution to the tasks. AI can relate to everything starting from software programs for playing chess and ending with speech recognition systems like Siri or Amazon Alexa voice assistant, which can perceive human speech and give answers to the questions. 

Machine learning is an application of AI and application of the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions, relying on patterns and inference instead. In simple words, ML is a technique by dint of which a computer does not just use a pre-written algorithm but learns how to solve the problem itself. 

What is more, ML-application not only solves the tasks but also records the features of the solution for creating your own experience, simplifying and clarifying the next steps. Machines just receive data known as "training data" and “learn” from it. Then ML allows applying the knowledge gained during training on large data sets.

Unlike programs with manually encoded instructions for performing specific tasks, machine learning allows the system to learn independently recognize patterns and make predictions. 

An example - a chess application studies the opponent’s behavior and takes it into account for the further reorganization of tactics.

Deep learning is a smaller element. It is a more specific application of AI, or more precisely a subset of machine learning and one of its algorithms. Right now DL is the most popular ML-algorithm that refers to a technique for creating an AI-powered layered neural network, much like a simplified replica of the human brain. DL gives computers the ability to solve more complex problems than other ML algorithms. 

An example - during a game of chess, a neural network is trained predominantly.

What is a neural network?

Neural networks, like biological ones, are composed of neurons. In machines, the neurons are virtual — basically bits of code running statistical regressions. String enough of these virtual neurons together and you get a virtual neural network.

A neural network is a learning system, that is, it operates not only on the basis of given algorithms but also based on our own experience. A neuron in deep learning can be thought of as a “black box” with many inlets and one outlet. At the input, the neuron receives signals and forms an output based on them. 

We know from biology that our ability to learn is based on the unique properties of a brain of 80 billion neurons. The collective work of these cells now allows you to understand what I am telling now. And neural network algorithms are trying to build a model of this process, although implausible biologically, but inspired by the laws of nature. 

In these programs, calculations are made by a network consisting of separate elements that process and transmit information to each other. In the process of spreading through the network, information changes - we call this process learning.

Brief History: Deep Learning Revolution and Neural Networks

If you search for deep learning in Wikipedia, you will notice this super hot word is far away from being new. Actually this technology was introduced to the machine learning community by Rina Dechter in 1986, and to artificial neural networks by Igor Aizenberg and colleagues in 2000, in the context of Boolean threshold neurons.

So yes, the roots of deep learning are decades old, but the term “deep learning” nor this approach was so popular before 2012. To the greatest extent, major breakthroughs occur after research and hard work of three scientists that is now well known as fathers of the Deep Learning:

Yoshua Bengio, Geoffrey Hinton and Yann LeCun - fathers of the Deep Learning

Okay then, I believe the history of DL development is something every tech- enthusiast should know. 

Let’s start with Geoffrey Hinton, he has been advocating for ML-approach to AI since the early 1980s. But, in the 80s everything was not so easy with that. That’s why more important breakthroughs occurred later when Hinton, LeCun, and Bengio have worked together

LeCun performed postdoctoral work under Hinton’s supervision, and LeCun and Bengio worked together at Bell Labs beginning in the early 1990s. Even while not working together, there is a synergy and interconnectedness in their work, and they have greatly influenced each other. 

Together they recognized the importance of building deep networks using many layers. The conceptual foundations and engineering advances laid by them over a 30-year period were significantly advanced by the prevalence of powerful graphics processing unit (GPU) computers, as well as access to massive datasets. These and other factors led to leap-frog advances in technologies like computer vision, speech recognition and more.

In March 2019, all together they were awarded the Turing Award for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.

Part II: How Everything Works?

Now it’s time to see the magic. 

So the central thing here is - Deep Learning uses a Neural Network to imitate animal intelligence. For doing this, Deep Learning is actively used during interaction with multilayer networks created by artificial means. 

What layers? Any neural network consists of many layers. Each next layer creates new (more complex) features, based on those that the previous layer gave it.

The architecture of a neural network can consist of many layers - information processing is divided into many stages. This is where the “deep” came from, by the way. In general, there are three types of layers of neurons in a neural network: 

• Input Layer: Input variables, sometimes called the visible layer.
• Hidden Layers: Layers of nodes between the input and output layers. There may be one or more of these layers.
• Output Layer: A layer of nodes that produce the output variables.

For example, it is necessary for the computer to recognize the cat in the photo. We collect data - millions of photos of cats - and give (feed) this data to the algorithm.

There can be a lot of layers, but imagine that to solve the problem, you need only 4. Each input of the first layer of neurons receives an incoming pixel of the picture.

In this case, each next layer combines information obtained at previous levels:

• 1 - the first layer of neurons can only recognize lines, points, and circles. When it understands where these objects are in the photo, it passes the information to the next layer. 
• 2 - based on this data, the algorithm determines that the next layer will be able to distinguish between triangles and squares, for example, to understand where are the kitten’s ears. 
• 3 - when the third layer finds out about this, he is already able to understand where the kitten’s head is in the image, and where is the body. 
• 4 - combining all the information received, the 4th layer of neurons understands that in front of it is an image of a kitten. So, the training was successful.

But how this magic works on?

Each connection between neurons is associated with a weight. This weight dictates the importance of the input value. The initial weights are set randomly. Predicting the cat on the photo, cat's features are one of the heavier factors. Hence, the cat's features neuron connections will have a big weight.

Each neuron has an Activation Function. These functions are hard to understand without mathematical reasoning. Simply put, one of its purposes is to “standardize” the output from the neuron. Once a set of input data has passed through all the layers of the neural network, it returns the output data through the output layer.

How to train a neural network?

So, we already know how a neural network works. Now we need to understand how the results are calculated. The main difficulty is to assign the right value for each connection in the neural network, and that’s why it needs to be trained.

The neural network processes all the data and sets the value to each neuron until it comes to the right conclusions about each element on which the data is collected. At the end of this stage, the value of each element becomes constant, and the neural network can more accurately give predictions.

 traditional machine learning vs deep learning 

While traditional machine learning algorithms are linear, deep learning algorithms are stacked in a hierarchy of increasing complexity and abstraction. To understand it better, imagine a toddler whose first word is a dog. The toddler learns what a dog is (and is not) by pointing to objects and saying the word dog. The parent says, “Yes, it is a dog,” or, “No, it is not a dog.” 

As the toddler continues to point to objects, he becomes more aware of the features that all dogs possess. What the toddler does, without knowing it, is to clarify a complex abstraction (the concept of dog) by building a hierarchy in which each level of abstraction is created with the knowledge that was gained from the preceding layer of the hierarchy.

Computer programs that use deep learning go through much the same process. Each algorithm in the hierarchy applies a nonlinear transformation on its input and uses what it learns to create a statistical model as output. Iterations continue until the output has reached an acceptable level of accuracy. The number of processing layers through which data must pass is what inspired the label deep.

Why did deep learning begin to spread recently? 

It is simply great in terms of accuracy when trained with a huge amount of data. This approach is significantly different from traditional programming and other machine learning methods. Deep learning can not only give results where other methods fail, but it also allows to build a more accurate model or reduce the time it takes to create it.

Are there any flaws?

Of course, they are everywhere and so far we are far from something ideal. As for deep learning, these shortcomings are certainly not so great. For example, accurate results require large computing power. Another drawback of deep learning is the difficulty in interpreting the resulting models.

At the current stage of development, it does not appear possible for deep learning to perform the same elaborate, adaptive thought processes of humans, however, the technology continues to evolve at quite a rate.

Part III: How is deep learning being used?

Deep networks allow you to build multi-stage information processing algorithms - it's like a layer cake that prepares itself. Many of today's most advanced machine learning systems use a neural network to process data. 

Popular uses today include:

• Speech Recognition

Both the business and academic worlds have embraced deep learning for speech recognition: Xbox, Skype, Google Now and Apple’s Siri and more.

• Image Recognition

Face recognition, automatic image captioning and scene description. (Google Lens, ScreenShop, FaceApp)

• Natural Language Processing

NLP deals with processing and analyzing written text. A specialization of text mining, this technique can be used to discover patterns in customer complaints, physician notes or news reports, to name a few. (Google Translate)

• Recommendation Systems

Deep learning can be used to enhance recommendations in complex environments such as music interests or clothing preferences across multiple platforms. (Amazon, Netflix, Spotify)

What has already been developed?

The largest number of projects today involve deep learning to recognize images and determine audio recordings, although there are already the first programs for diagnosing diseases. Neural networks are used by Google for the application of translating text from pictures. Using Deep Learning, it is easier to determine the presence of letters in photographs and their outline, and then the program translates the resulting text.

One of the most interesting projects is DeepFace, it was developed to determine facial features. Today the accuracy of the program reaches 97.25% - the same accuracy that humans have.

Another interesting project - WaveNet launched in 2016. It is a system for simulating a human voice. In order to achieve quality training, millions of minutes of voice conversations with the Okay Google were downloaded into the system. After the entire training cycle, the machine independently drew up a proposal, everywhere placed the correct stresses, a characteristic accent, without any inappropriate pauses.

Final word

Neural networks taught to recognize tumors in x-rays, play go and chess, create music and texts. And this is not the limit. We should definitely be interested in new technologies because this is our future.

Thanks for reading and feel free to share if you think it deserves to be!

Relevant links to go deeper:

• Introduction to Convolutional Neural Networks by Jianxin Wu and Yann LeCun’s
• Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville
• Grokking Deep Learning by Andrew W. Trask
• Machine Learning Yearning by Andrew Ng
• A practical guide to Deep Learning in 6 months
• Efficient BackProp by Yann LeCun and others 

Inspired to learn more about AI, ML & Data Science? 

Welcome to my Instagram and Medium blog

Cheers!