Different Types of Machine Learning Algorithms

Machine Learning
Presented By
Dr. Md. Zahid Hasan
Associate Professor, CSE, DIU

Linear Regression
• Linear Regression is the supervised Machine Learning model in which
the model finds the best fit linear line between the independent and
dependent variable i.e it finds the linear relationship between the
dependent and independent variable.
• The core idea is to obtain a line that best fits the data. The best fit line is
the one for which total prediction error (all data points) are as small as
possible. Error is the distance between the point to the regression line.
3

Types of Linear Regression
• Linear Regression is of two types: Simple and
Multiple. Simple Linear Regression is where
only one independent variable is present and
the model has to find the linear relationship of
it with the dependent variable
• Whereas, In Multiple Linear Regression there
are more than one independent variables for
the model to find the relationship.
4

Equation of Simple Linear Regression
• For a set of data points: (xi,yi), we can write the equation of
the line as:
where yi is the predicted y-value, not the actual y-values of our points.
• The gradient - m and y-intercept - c are called fit parameters. By
using the method of linear regression (also called the method of
least squares fitting), we can calculate the values for the two
parameters and plot our line of best fit.
• Calculate Slope and Intercept by using the formula
5
m=
𝑛 𝑥𝑦 − 𝑥 𝑦
𝑛 𝑥2 −( 𝑥)2

Dataset for Simple Linear Regression
Years Experience Salary
1 1.1 39343.00
2 1.3 46205.00
3 1.5 37731.00
4 2.0 43525.00
5 2.2 39891.00
6

Simple Linear Regression Solution
SL.
Years
Experience (x)
Salary (y) Xy 𝒙𝟐
1 1.1 39343.00 43277.3 1.21
2 1.3 46205.00 60066.5 1.69
3 1.5 37731.00 56596.5 2.25
4 2.0 43525.00 87050.0 4.0
5 2.2 39891.00 87760.2 4.84
𝑥 = 8.1 𝑦 = 206,695 𝑥 = 334,750.5 𝑥2
= 13.99
7
Mean of x ;
x̅ = 1.62
Mean of y;
y̅ = 41339.0

8
m =
𝑛 𝑥𝑦 − 𝑥 𝑦
𝑛 𝑥2 −( 𝑥)2
=
5𝑋334,750.5 − 8.1𝑋206,695.0
5𝑋13.99 − 65.61
= -109.91
c = y̅ - mx̅
= 41339.0 – (-109.91 X 1.62)
= 41517.05

9
In this example, of an individual person years of
experience was 5 years, we would predict his
Expected salary to be:
y = mx + c
= -109.91 X 5 + 41517.05
= 40967.5
In this simple linear regression, we are examining
the impact of one independent variable on the
outcome.

Multiple Linear Regression
• Equation of Multiple Linear Regression, where
bo is the intercept, b1,b2,b3,b4…,bn are
coefficients or slopes of the independent
variables x1,x2,x3,x4…,xn and y is the
dependent variable.
10

Dataset for Multi variable Regression
Area Bedrooms Age Price
2600 3 20 550000
3000 4 15 565000
3200 3 18 610000
3600 3 30 595000
4000 5 8 760000
11

Multi variable Regression solution
12
Mean of x̅1, x̅2, x̅3:
x̅1 = 3280; x̅2 = 3.6;
x̅3 = 18.2
Mean of y̅ = 616,000

Multi variable Regression Solution
13
m1 =
𝑛 𝑥1𝑦 − 𝑥1 𝑦
𝑛 𝑥12 −( 𝑥1)2
=442.29
m3 =
𝑛 𝑥32 −( 𝑥3)2
= -6507.01
m2 =
𝑛 𝑥22 −( 𝑥2)2
= 74062.5
c = y̅ - m1x̅1 – m2x̅2 – m3x̅3
= 616000 – 442.29 X 3280 – 74062.5 X 3.6 - (-
6507.01 X 18.2)
= -982908.62

Multi variable Regression Solution
15
m1 =442.29
m3 = -6507.01
m2 = 74062.5
c = -982908.62
Given these home prices find out price of a home that
has:
1. 3000 sqr ft area, 3 bedrooms, 40 years old.
2. 2500 sqr ft area, 4 bedrooms, 5 years old.
1. 442.29 X 3000 + 74062.5 X 3 + (-6507.01) X 40 + (-
982908.62)
= 305868.30
2. 442.29 X 2500 + 74062.5 X 4 + (-6507.01) X 5 + (-
982908.62)
= 386531.38

Library Used in Program
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn import linear_model
from sklearn.model_selection import train_test_split
import seaborn as sns
from sklearn import metrics
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
16

Data frame and Array
• #Salary Dataset
• # Generates data frame from csv file
df = pd.read_csv("F:/AI and Machine learning
Book/Coding/Salary_Data.csv")
• # Turning the columns into arrays
x = df["YearsExperience"].values
y = df["Salary"].values
17

Plot the data in Graph
• # Plots the graph from the above data
plt.figure()
plt.grid(True)
plt.plot(x,y,'r.')
YearsExperience Salary
1.1 39343
1.3 46205
1.5 37731
2 43525
2.2 39891
2.9 56642
3 60150
3.2 54445
3.2 64445
3.7 57189
3.9 63218
4 55794
18

Calculate Gradient and Intercept
• Independant variable or features
x = x.reshape(-1,1)
• Dependant variable or labels
y = y.reshape(-1,1)
• Seperates the data into test and training sets
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size = 0.2)
• Plotting the training and testing splits
plt.scatter(X_train, y_train, label = "Training Data", color = 'r')
plt.scatter(X_test, y_test, label = "Testing Data", color = 'b')
plt.legend()
plt.grid("True")
plt.title("Test/Train Split")
plt.show()
19

Define Linear Regression
• # Defining our regressor
regressor = linear_model.LinearRegression()
• # Train the regressor
fit = regressor.fit(X_train, y_train)
20

Gradient and Intercept
• # Returns gradient and intercept
print("Gradient:",fit.coef_)
print("Intercept:",fit.intercept_)
21

Predicted Lines
• # Predicted values
y_pred = regressor.predict(X_test)
• # Plot of the data with the line of best fit
plt.plot(X_test,y_pred)
plt.plot(x,y, "rx")
plt.grid(True)
22

Compare Predicted and Actual Value
• #Converts predicted values and test values to
a data frame
df = pd.DataFrame({"Predicted": y_pred[:,0],
"Actual": y_test[:,0]}) Predicted Actual
0 60820.440334 57189.0
1 54176.807620 60150.0
2 56074.988396 54445.0
3 115867.682821 116969.0
4 39940.451805 37731.0
5 125358.586698 121872.0
23

Determine Score of the model
• # Determines a score for our model
score = regressor.score(X_test, y_test)
print(score)
24

Read Dataset
• Converts advertising csv to a data frame
df = pd.read_csv("F:/AI and Machine learning
Book/Coding/advertising.csv")
df
26

Drop Column and Split Dataset
• In the following code cell, we can see that Sales is dropped from df
so that only independent variables x remain. Now we specify Sales
as y since it is the dependent variable and we need to reshape it
because it consists of only one column
• Independent variables
X = df.drop("Sales",axis=1)
• Dependent variable
y = df["Sales"].values.reshape(-1,1)
• Splitting into test and training data
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
27

Use Linear Regression
• Defining regressor
regressor = linear_model.LinearRegression()
• Training our regressor
fit = regressor.fit(X_train,y_train)
• Predicting values
y_pred = fit.predict(X_test)
28

Compare predicted and Actual value
• Comparing predicted against actual values
df = pd.DataFrame({"Predicted": y_pred[:,0], "Actual": y_test[:,0]})
df
29

Plot with Best fitted line
• Plot of the data with the line of best fit
plt.plot(X_test,y_pred)
plt.plot(X,y, "rx")
plt.grid(True)
30

Score of the model
• # Scoring our regressor
fit.score(X_test,y_test)
Accuracy=0.9291555806063022
31

Save the model in a file
• import pickle
• filename = '/content/drive/MyDrive/Summer
2022/MSC/Linear_Regression/finalized_model
.sav‘
• pickle.dump(fit, open(filename, 'wb'))
33

Load the saved model
• loaded_model = pickle.load(open(filename, 'r
b'))
• loaded_model.coef_
• loaded_model.intercept_
• loaded_model.predict([[5000]])
34

R^2 Square value
from sklearn import metrics
print('Model R^2 Square value', metrics.r2_score(y_test, y_pred))
• Model R^2 Square value 0.9291555806063022
• The Goal of Linear Regression is to find out the best hypothesis which
maximize the R^2 Square value.
• The coefficient of determination, or R^2, is a measure that provides
information about the goodness of fit of a model. In the context of
regression it is a statistical measure of how well the regression line
approximates the actual data. It is therefore important when a statistical
model is used either to predict future outcomes or in the testing of
hypotheses.
35

• Input x=[1,2,3,4,5]
• Output y=[5,7,9,11,13]
• Derive an Equation (Prediction Function)
y=2x+3
• Area=[2600,3000,3200,3600,4000]
• Price= [550K, 565k, 610k, 680k, 725k]
• Derive Equation (Prediction Function)
Price= 135.78*Area+180616.43
36

Price vs Area using Linear Regression
(Multiple Best Fit Line)
37

Calculate error from Data Points
38

Fixed Size Step to reach global Minima
42

Small steps (learning rate)
43

Partial Derivative of Mean Square
Error
44

Updating b using Partial Derivative
45

Implementation
• import numpy as np
• import matplotlib.pyplot as plt
• def gradient_descent(x,y):
m_curr = b_curr = 0
rate = 0.01
n = len(x)
plt.scatter(x,y,color='red',marker='+',linewidth='5')
for i in range(10000):
y_predicted = m_curr * x + b_curr
plt.plot(x,y_predicted,color='green')
md = -(2/n)*sum(x*(y-y_predicted))
yd = -(2/n)*sum(y-y_predicted)
m_curr = m_curr - rate * md
b_curr = b_curr - rate * yd
46

Plotting Gradient Descent
• x = np.array([1,2,3,4,5])
• y = np.array([5,7,9,11,13])
• gradient_descent(x,y)
47

Different Types of Machine Learning Algorithms

Recommended

More Related Content

Similar to Different Types of Machine Learning Algorithms (20)

Recently uploaded (20)

Different Types of Machine Learning Algorithms