Machine learning algorithms for data mining
Download as pptx, pdf0 likes309 views
This slide is prepared for a course of Dept. of CSE, Islamic Univresity of Technology (IUT). Course: CSE 4739- Data Mining
More Related Content
What's hot (18)
Similar to Machine learning algorithms for data mining (20)
More from Ashikur Rahman (10)
Recently uploaded (20)
Machine learning algorithms for data mining
- 1. Machine Learning Methods for Data Mining Based on- Data Mining: Concepts and Techniques Han, Kamber & Pei A.B.M. Ashikur Rahman Asst. Professor, Dept. of CSE, IUT
- 2. Data Mining Knowledge Discovery from Data (KDD) process steps- • Data Cleaning • Data Integration • Data Selection • Data Transformation • Pattern Mining • Pattern Evaluation • Knowledge Representation e.g.- Frequent itemsets, Association rule (Strong/week)
- 3. 3 Supervised vs. Unsupervised Learning • Supervised learning (classification) • Supervision: The training data (observations, measurements, etc.) are accompanied by labels indicating the class of the observations • New data is classified based on the training set • Unsupervised learning (clustering) • The class labels of training data is unknown • Given a set of measurements, observations, etc. with the aim of establishing the existence of classes or clusters in the data
- 4. 4 Classification vs. Numeric Prediction • Classification • predicts categorical class labels (discrete or nominal) • classifies data (constructs a model) based on the training set and the values (class labels) in a classifying attribute and uses it in classifying new data • Numeric Prediction • models continuous-valued functions, i.e., predicts unknown or missing values • Typical applications • Credit/loan approval: • Medical diagnosis: if a tumor is cancerous or benign • Fraud detection: if a transaction is fraudulent • Web page categorization: which category it is Prediction Problems:
- 5. 5 Classification—A Two-Step Process • Model construction: describing a set of predetermined classes • Each tuple/sample is assumed to belong to a predefined class, as determined by the class label attribute • The set of tuples used for model construction is training set • The model is represented as classification rules, decision trees, or mathematical formulae • Model usage: for classifying future or unknown objects • Estimate accuracy of the model • The known label of test sample is compared with the classified result from the model • Accuracy rate is the percentage of test set samples that are correctly classified by the model • Test set is independent of training set (otherwise overfitting) • If the accuracy is acceptable, use the model to classify new data • Note: If the test set is used to select models, it is called validation (test) set
- 6. 6 Process (1): Model Construction Training Data NAME RANK YEARS TENURED Mike Assistant Prof 3 no Mary Assistant Prof 7 yes Bill Professor 2 yes Jim Associate Prof 7 yes Dave Assistant Prof 6 no Anne Associate Prof 3 no Classification Algorithms IF rank = ‘professor’ OR years > 6 THEN tenured = ‘yes’ Classifier (Model)
- 7. 7 Process (2): Using the Model in Prediction Classifier Testing Data NAME RANK YEARS TENURED Tom Assistant Prof 2 no Merlisa Associate Prof 7 no George Professor 5 yes Joseph Assistant Prof 7 yes Unseen Data (Jeff, Professor, 4) Tenured?
- 8. Classification Methods • Decision Tree Induction • Naïve Bayesian Classification • Rule based Classification • Bayesian Belief Network • Support Vector Machine (SVM) etc.
- 9. 9 What is Cluster Analysis? • Cluster: A collection of data objects • similar (or related) to one another within the same group • dissimilar (or unrelated) to the objects in other groups • Cluster analysis (or clustering, data segmentation, …) • Finding similarities between data according to the characteristics found in the data and grouping similar data objects into clusters • Unsupervised learning: no predefined classes (i.e., learning by observations vs. learning by examples: supervised) • Typical applications • As a stand-alone tool to get insight into data distribution • As a preprocessing step for other algorithms
- 10. 10 Clustering for Data Understanding and Applications • Biology: taxonomy of living things: kingdom, phylum, class, order, family, genus and species • Information retrieval: document clustering • Land use: Identification of areas of similar land use in an earth observation database • Marketing: Help marketers discover distinct groups in their customer bases, and then use this knowledge to develop targeted marketing programs • City-planning: Identifying groups of houses according to their house type, value, and geographical location • Earth-quake studies: Observed earth quake epicenters should be clustered along continent faults • Climate: understanding earth climate, find patterns of atmospheric and ocean • Economic Science: market resarch
- 11. 11 Clustering as a Preprocessing Tool (Utility) • Summarization: • Preprocessing for regression, PCA, classification, and association analysis • Compression: • Image processing: vector quantization • Finding K-nearest Neighbors • Localizing search to one or a small number of clusters • Outlier detection • Outliers are often viewed as those “far away” from any cluster
- 12. Quality: What Is Good Clustering? • A good clustering method will produce high quality clusters • high intra-class similarity: cohesive within clusters • low inter-class similarity: distinctive between clusters • The quality of a clustering method depends on • the similarity measure used by the method • its implementation, and • Its ability to discover some or all of the hidden patterns 12
- 13. Measure the Quality of Clustering • Dissimilarity/Similarity metric • Similarity is expressed in terms of a distance function, typically metric: d(i, j) • The definitions of distance functions are usually rather different for interval- scaled, boolean, categorical, ordinal ratio, and vector variables • Weights should be associated with different variables based on applications and data semantics • Quality of clustering: • There is usually a separate “quality” function that measures the “goodness” of a cluster. • It is hard to define “similar enough” or “good enough” • The answer is typically highly subjective 13
- 14. Major Clustering Approaches (I) • Partitioning approach: • Construct various partitions and then evaluate them by some criterion, e.g., minimizing the sum of square errors • Typical methods: k-means, k-medoids, CLARANS • Hierarchical approach: • Create a hierarchical decomposition of the set of data (or objects) using some criterion • Typical methods: Diana, Agnes, BIRCH, CAMELEON • Density-based approach: • Based on connectivity and density functions • Typical methods: DBSACN, OPTICS, DenClue • Grid-based approach: • based on a multiple-level granularity structure • Typical methods: STING, WaveCluster, CLIQUE 14