(Pt2) How to easily code a Linear Regression model using TensorFlow and python!

Understanding machine learning might be easier than you think. Last week we talked about the basics of machine learning algorithms. Now we can get a little bit deeper in the use of the linear regression model. So let's get started:

As I wrote in my last post, most of machine learning algorithms can be summarized into 4 easy steps:

data -> model -> objective function -> optimization algorithm

After getting the data, you choose the model that best fits your data. And as we're talking about the linear regression model, let see how it works.

The linear regression model

The linear model is exactly the same linear function we learn in high school, where X are the points entered, and Y are the predicted values associated with each data entered. m and b are called parameters in the world of machine learning, and they'll vary automatically by the optimization algorithm, so we don't have to worry about them. You can think of them as being values that are adjusted automatically while our algorithm is being "optimized" until they reach values that create the best predicting line for your data.

Sizing of the data

If you have only one input in X, you'll have only one output in Y, just like we learned in high school. But what if we have more than one input for each output? Or many outputs for each input? Well, in these cases, surprisingly, the formula is still the same! Yes, you don't have to change anything in the formula. However, what you get to change is that instead of entering single values and multiplying single values, you enter a matrix of values and multiply matrices of values. If you work with matrices, you don't have to change the formula. But worry not, you won't actually have to get deeper into the mathematics, most of the things are taken care for you via software. What you do have to know though, are the sizes of each matrix you multiply in the formula. And they go like this:

1. n -> The number of observations
2. m -> The number of output variables
3. k -> The number of input variables

So, if you take the output matrix Y for example, it's size will be of nxm (n rows x m columns). Of course if you're working with multiple variables like this, when plotting the graph it's going to generate a 3D graph. But you don't have to worry about it for now.

For this tutorial we'll only see a simple linear regression, where the number of n, m and k equals to 1. However, I wanted to show you that it's possible to work with multiple variables as well, and you'll probably need this knowledge when predicting using linear regression in the future.

The mean square error objective function

We already have the data and the model, what's the next step? You're right, the objective function.

The objective function is the function that will calculate the error between the model you created, and the actual values.

Using the linear model, you are trying to create a line which will predict the output of many data entered. So when you create your line using the linear regression formula, the objective function will check if this line created matches the actual values outputted. This function will tell you how big is the error between your line and the actual values that should be outputted.

There're some functions that can calculate the error, but for the linear model, the mean squared error is one that works the best. This is the formula for the MSE(Mean Square Error):

Yeah, horrible right? Fortunately you don't have to understand it in order to use it. TensorFlow has a function that calculates it for you, so don't worry.

The Gradient Descent Optimization Algorithm

Now that we know the error generated between our predictive model and the actual values, we just have to adjust our model to minimize this error generated the most we can.

How do we do it? Adjusting the parameters in our model. In a linear regression model, the parameters are the m and the b of the linear formula. Remember I told you that the optimization algorithm would adjust them for you automatically? It does that by following a mathematical algorithm that chooses the best values for m and b in order to minimize the error.

This algorithm, well, is hard to understand an math heavy. But guess what, TensorFlow has a function that does that automatically for you!

One of the optimization algorithms most used for linear regression models is the gradient descent.

This is a good representation of how the algorithm works. On the left we have a graph, where we have the values of m and b, and the error (generated by the objective function). The algorithm tries to find the values of m and b where the error is the lowest, and you can see that on the right, the line is adjusted until it fits the data points the best. And this is how we adjust our line to become a good predictive model!

Conclusion

In this tutorial we finished learning everything we needed to know about the linear regression model. In the next tutorial (I promise), we gonna get our hands dirty with the actual coding.