The 8 Step Data Mining Process

THE DATA MINING
PROCESS
LEARNING OBJECTIVES:
• PROVIDE AN OVERVIEW OF THE EIGHT STEPS IN THE DATA MINING
PROCESS.
• IDENTIFY THE ISSUES INVOLVED IN DEFINING A DATA MINING PROBLEM.
• DETERMINE WHEN TO USE AND NOT TO USE DATA MINING
• EXPLAIN HOW TO CONDUCT AND EXPERIMENT TO DETERMINE WHETHER
MORE DATA IS NEEDED
By Obakeng Brian Pheelwane & Marc Berman – Group 14

8 STEP DATA MINING PROCESS
• Defining the problem
• Collecting data
• Preparing data
• Pre-processing
• Selecting an algorithm
and training parameters
• Training and testing
• Iterating to produce
different models
• Evaluating the final
model
- The next three steps involve collecting the required data, preparing
and pre-processing the data before a data mining technique
could be applied.
-This defines the objective of the whole data mining process.
- Select a model and perform training and testing to evaluate if the model is good.
- Because one does not know how an algorithm will perform on a data set, one
needs to try different models and compare each model against each other
model.
- Due to the previous set, many iterations may be required for all steps in order to
determine the optimal model.
- The best model is selected based on the estimated accuracy. This model is then
used for future predictions.

CONTINUED…
Defining the problem:
• One needs to determine which problems are suitable for data-driven modelling.
• How does one evaluate the results?
• Is it a classification or estimation problem?
• What are the inputs and outputs required for solving the problem?
When TO DO data mining:
• When no good existing solution exists & the problem has the following characteristics:
 Lots of data
 The problem is not well understood
When NOT to do data mining:
• When the problem:
 Has a complete, closed-form mathematical solution.
 It is well understood and has a good analytical or rule-based solution.
 The problem can be characterised as an input-to-output relationship
 Existing models have strong and possibly erroneous assumptions

CONTINUED…
How do you evaluate the results?
• What level of accuracy would be considered successful?
• How will you benchmark the performance of a developed solution?
• What existing alternatives will you compare against?
• What kind of data will be used to evaluate the various models?
• What will the models be used for and how well do they support that purpose?
Classification or Estimation?
Discrete outputs = classification problem
Continuous outputs = estimation problem
Borderline outputs = can be either based upon the granularity of the outputs.

CLASSIFICATION VERSUS ESTIMATION
Classification:
In classification learning, the learning scheme is presented with a set of classified examples from which it is
expected to learn a way of classifying unseen examples.
In plain English, the main idea behind classification is it uses 2 values, it is either yes or no, 1 or 0. It may
belong to a single class not both. The algorithm will then learn how to predict future unseen data based upon
the training data.
Estimation:
In estimation, the classification of the data is not based on an absolute value (IE 0 or 1) but rather on real
numbers between 0 and 1. So what if the value sits between two classes? For example, 0,5 could fall into
either class however if the classes state 0-0,49 and 0,5 to 1 then it would inevitably fall into the later class.
When the model is first designed, a set of predefined classes are laid out in order to prevent a data point from
lying between two or more categories.

WHAT ARE THE INPUTS AND OUTPUTS?
An example: The inputs and outputs when classifying loan application at a bank.
Outputs:
The outputs could possibly be “high risk” or “low risk”
Inputs:
The inputs may current salary, outstanding liabilities, bank account balances, other income, number of years
employed.

IMPORTANT ISSUES
Causal and Non-Causal outputs:
Causal attributes occur when one attribute causes another. Meaning it is used to predict another attribute or it
is included in the calculation. It is important to avoid using non-causal attributes as they produce a model not a
representation of the future data.
Inputs:
The inputs must contain enough information to be able to generate the desired output. If the inputs contain
insufficient data the accuracy of the final model will inevitably decrease.
Data set:
The data chosen must be an accurate representation of the future data set. If it is not the accuracy when trying
to predict the future will produce undesired outputs.

HOW MUCH DATA IS ENOUGH
Key: The amount of data required depends on the problem
complexity as well as the amount of noise in the data.
Each learning algorithm follows its own learning curve, the accuracy
increases as the data size increases, however when the algorithm
reaches its optimal performance further increase of the data set
CANNOT improve performance.
Figure 1:
Algorithm A reaches optimal performance sooner than Algorithm B,
however the accuracy cannot increase after a certain point.
Note: One needs to experiment to determine when the algorithm
reaches it’s optimal performance. Can be determined by:
• Using small portions of the training set to and gradually increasing the
size until the accuracy level stabilises.
Figure 1:

WHAT WE HAVE LEARNED:
• The 8 steps of data mining as well as the need for iteration in order to produce a satisfactory
model.
• The difference between estimation and classification
• When to do data mining and when not to
• Important issues when defining a problem

REVIEW QUESTIONS
1. Explain how are you going to decide whether a given problem is suitable for a data
mining solution.
Given a problem , I am going to check if there is no good existing solution and the problem has the
following characteristics:
• Lots of data
• The problem is not well understood
• The problem can be characterised as an input-to-output relationship
• Existing models have strong and possibly erroneous assumptions

2. Suppose the data provided is the last promotional mail-out records which consist of
information about each of the 100000 customers (name, address, occupation, salary)
and whether each individual customer responded to the mail (i.e., an attribute indicating
“yes” or “no”. You are asked to produce a data mining solution, that is, a model
describing the characteristics of customers who are likely, as well as unlikely, to
respond to the promotional mail-out. The company could then use this model to target
customers who are likely to respond to the next promotional mail-out for the same
product.
Discuss the following issues:
• Is this problem suitable for data mining solution?
Yes
This problem is suitable for data mining solution because there is no good existing solution and it has a lot
of data, that can be easily characterised as an input-to-output relationship.

• Does the information above give us a classification or estimation problem? Justify your answer
Classification problem, the model required is to be used to predict whether a customer is a repeat buyer or not.
i.e. two classes of customers.
• What are the inputs and output?
The inputs are the attributes: name, address, occupation, salary;
The outputs is an attribute called: “Repeat Buyer” with the labels “yes” or “no”
• What is the alternative to producing a model?
As the task is to select a subset of customers to send the promotional mail-out to, instead of building a model
in order to identify the customers, one can simply perform a random selection of customers from all the
available customers
• How you will use the data for training a model and evaluating the model?
In order to evaluate the trained models using test data, the given data set of 100000 customers can be split
into two subsets; one for the training and one for the testing. There is no need to use more elaborate
evaluation method such as 10-fold cross validation method, as the data set is big enough and the reserved for
testing is unlikely to degrade the predictive accuracy of the trained model.

3. Let say you are given a set of training data with 50% class “positive” and 50% class
“negative”, and you have explored several models and selected the best model. You can
now use the best model to do future prediction. Now, you are informed that the future data
you are going to get is likely to have the following class distribution: 90% class “positive”
and 10% class “negative”.
Would you go ahead to use the best model to do prediction for all future data? Provide a reason for your
answer. In the case that your answer is no, you shall also provide an alternative solution to do prediction for
the future data.
The training data is not a representative of the testing data in terms of class distribution; the one has a 1:1
ratio and another has 9:1 ratio. This is why one should not use the model from a data set not representative of
the testing data to do prediction for future data.
If one does, then it is likely to lead to poor performance i.e. high error rate.

The same model applied to 9:1 ratio testing data will perform a lot worse, as 80% of the time the model will
predict a class “positive” regardless of the input. The simple model will have accuracy of 90% on the 9:1 ratio
testing data.
One should be aware of how some models allow one to adjust outputs to match the class distribution of the
testing data. Only with this adjustment, than one can use the model trained from different class distribution to
be applied.
4. How does one decide whether to collect more data or not in a non-time series data
mining task?
When there is a small portion of data to mine, one should collect more data.
Where the is a lot of data one should not collect more data is there is sufficient data to mine.
In both situations time is not a factor, everything is based on the quantity of data required to mine a data
mining task.

The 8 Step Data Mining Process

Recommended

More Related Content

What's hot (20)

Similar to The 8 Step Data Mining Process (20)

Recently uploaded (20)

The 8 Step Data Mining Process

Editor's Notes