SlideShare a Scribd company logo
VC Dimension in Machine Learning
Dr. Varun Kumar
Dr. Varun Kumar Lecture 18 1 / 10
Outlines
1 General Classification Problem
2 Usage of VC dimension in ML
3 Introduction to Vapnik-Chervonenkis (VC) Dimension
4 How to Determine VC Dimension for a Given Classifier or Hypothesis?
5 References
Dr. Varun Kumar Lecture 18 2 / 10
General classification problem
1 Always look for test error along with the training error.
2 Improving on training error does not improve the test error.
3 Increase in machine capacity may give the poor performance.
Is there any equation that relates the training and test error ?
Dr. Varun Kumar Lecture 18 3 / 10
Usage of VC dimension in ML
Model complexity determines the performance/cost on both the training
and test sets.
P

Test error ≤ Training error +
r
h(log(2N/h) + 1) − log η/4
N

= 1 − η
Note: Above expression shows the upper bound of test error with
probability 1 − η.
h→ VC dimension
h measure the power
h does not depend on the choice of training set
N → Total number of training sample
For reducing the residual, h → low or N → high
Test error ≤ Training error + Penalty(Complexity)
.
Dr. Varun Kumar Lecture 18 4 / 10
Continued–
⇒ Let us our training data are iid from some distribution fX (x).
⇒ Types of risk
(i) Risk R(θ)→ Long term observation→ Test observation
R(θ) = Test error = E[δ(c 6= ĉ(x; θ))]
(ii) Empirical risk Remp
(θ)→ Finite sample observation→ Training
observation
Remp
(θ) = Training error =
1
m
X
i
[δ(c(i)
6= ĉ(i)
(x; θ))]
Dr. Varun Kumar Lecture 18 5 / 10
Introduction to Vapnik-Chervonenkis (VC) Dimension
Key features:
⇒ VC dimension is a measure of the capacity (complexity, expressive
power, richness, or flexibility) of a set of functions.
⇒ It learns by a statistical binary classification algorithm.
⇒ It is defined as the cardinality of the largest set of points that the
algorithm can shatter.
Cardinality refers to the size of set. Ex- A = {1, 4, 6}, cardinality
|A| = 3
⇒ The capacity of a classification model is related to how complicated it
can be.→ Overfitting
VC dimension of a set-family
Let H be a set family (a set of sets) and C a set.
H ∩ C := {h ∩ C | h ∈ H}.
Dr. Varun Kumar Lecture 18 6 / 10
Relationship between risk and model complexity
Dr. Varun Kumar Lecture 18 7 / 10
How to determine VC dimension for a given classifier or hypothesis?
1 General point setting:
Statement: In a n−dimensional feature space a set of m points (m  n) is
in general position if and only if no subset of (m + 1) points lie on the
(n − 1) dimensional hyperplane.
Dr. Varun Kumar Lecture 18 8 / 10
2 Shattering:
Statement: A hypothesis H shatter m points in n− dimensional space if
all possible combinations of m points in n− dimensional space are
correctly classified.
Dr. Varun Kumar Lecture 18 9 / 10
References
E. Alpaydin, Introduction to machine learning. MIT press, 2020.
T. M. Mitchell, The discipline of machine learning. Carnegie Mellon University,
School of Computer Science, Machine Learning , 2006, vol. 9.
J. Grus, Data science from scratch: first principles with python. O’Reilly Media,
2019.
Dr. Varun Kumar Lecture 18 10 / 10
Ad

More Related Content

What's hot (20)

Perceptron (neural network)
Perceptron (neural network)Perceptron (neural network)
Perceptron (neural network)
EdutechLearners
 
Random forest
Random forestRandom forest
Random forest
Musa Hawamdah
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
mrizwan969
 
Back propagation
Back propagationBack propagation
Back propagation
Nagarajan
 
Decision Tree Learning
Decision Tree LearningDecision Tree Learning
Decision Tree Learning
Milind Gokhale
 
Presentation on supervised learning
Presentation on supervised learningPresentation on supervised learning
Presentation on supervised learning
Tonmoy Bhagawati
 
Types of Machine Learning
Types of Machine LearningTypes of Machine Learning
Types of Machine Learning
Samra Shahzadi
 
Support Vector Machines ( SVM )
Support Vector Machines ( SVM ) Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
Mohammad Junaid Khan
 
Token, Pattern and Lexeme
Token, Pattern and LexemeToken, Pattern and Lexeme
Token, Pattern and Lexeme
A. S. M. Shafi
 
Machine learning Lecture 2
Machine learning Lecture 2Machine learning Lecture 2
Machine learning Lecture 2
Srinivasan R
 
Overfitting & Underfitting
Overfitting & UnderfittingOverfitting & Underfitting
Overfitting & Underfitting
SOUMIT KAR
 
Learning set of rules
Learning set of rulesLearning set of rules
Learning set of rules
swapnac12
 
Unit I & II in Principles of Soft computing
Unit I & II in Principles of Soft computing Unit I & II in Principles of Soft computing
Unit I & II in Principles of Soft computing
Sivagowry Shathesh
 
Deep learning
Deep learningDeep learning
Deep learning
Ratnakar Pandey
 
Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning
Mohammad Junaid Khan
 
Machine learning clustering
Machine learning clusteringMachine learning clustering
Machine learning clustering
CosmoAIMS Bassett
 
Machine Learning-Linear regression
Machine Learning-Linear regressionMachine Learning-Linear regression
Machine Learning-Linear regression
kishanthkumaar
 
Analytical learning
Analytical learningAnalytical learning
Analytical learning
swapnac12
 
Grid based method & model based clustering method
Grid based method & model based clustering methodGrid based method & model based clustering method
Grid based method & model based clustering method
rajshreemuthiah
 
Decision tree
Decision treeDecision tree
Decision tree
R A Akerkar
 
Perceptron (neural network)
Perceptron (neural network)Perceptron (neural network)
Perceptron (neural network)
EdutechLearners
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
mrizwan969
 
Back propagation
Back propagationBack propagation
Back propagation
Nagarajan
 
Decision Tree Learning
Decision Tree LearningDecision Tree Learning
Decision Tree Learning
Milind Gokhale
 
Presentation on supervised learning
Presentation on supervised learningPresentation on supervised learning
Presentation on supervised learning
Tonmoy Bhagawati
 
Types of Machine Learning
Types of Machine LearningTypes of Machine Learning
Types of Machine Learning
Samra Shahzadi
 
Token, Pattern and Lexeme
Token, Pattern and LexemeToken, Pattern and Lexeme
Token, Pattern and Lexeme
A. S. M. Shafi
 
Machine learning Lecture 2
Machine learning Lecture 2Machine learning Lecture 2
Machine learning Lecture 2
Srinivasan R
 
Overfitting & Underfitting
Overfitting & UnderfittingOverfitting & Underfitting
Overfitting & Underfitting
SOUMIT KAR
 
Learning set of rules
Learning set of rulesLearning set of rules
Learning set of rules
swapnac12
 
Unit I & II in Principles of Soft computing
Unit I & II in Principles of Soft computing Unit I & II in Principles of Soft computing
Unit I & II in Principles of Soft computing
Sivagowry Shathesh
 
Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning
Mohammad Junaid Khan
 
Machine Learning-Linear regression
Machine Learning-Linear regressionMachine Learning-Linear regression
Machine Learning-Linear regression
kishanthkumaar
 
Analytical learning
Analytical learningAnalytical learning
Analytical learning
swapnac12
 
Grid based method & model based clustering method
Grid based method & model based clustering methodGrid based method & model based clustering method
Grid based method & model based clustering method
rajshreemuthiah
 

Similar to Vc dimension in Machine Learning (20)

Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)
VARUN KUMAR
 
13ClassifierPerformance.pdf
13ClassifierPerformance.pdf13ClassifierPerformance.pdf
13ClassifierPerformance.pdf
ssuserdce5c21
 
Understanding Blackbox Prediction via Influence Functions
Understanding Blackbox Prediction via Influence FunctionsUnderstanding Blackbox Prediction via Influence Functions
Understanding Blackbox Prediction via Influence Functions
SEMINARGROOT
 
14 ch ken black solution
14 ch ken black solution14 ch ken black solution
14 ch ken black solution
Krunal Shah
 
Introduction to Machine Learning Lectures
Introduction to Machine Learning LecturesIntroduction to Machine Learning Lectures
Introduction to Machine Learning Lectures
ssuserfece35
 
Lecture6 xing
Lecture6 xingLecture6 xing
Lecture6 xing
Tianlu Wang
 
Machine learning in science and industry — day 1
Machine learning in science and industry — day 1Machine learning in science and industry — day 1
Machine learning in science and industry — day 1
arogozhnikov
 
26 Ch. 3 Organizing and Graphing DataAssignment 2ME.docx
26     Ch. 3 Organizing and Graphing DataAssignment 2ME.docx26     Ch. 3 Organizing and Graphing DataAssignment 2ME.docx
26 Ch. 3 Organizing and Graphing DataAssignment 2ME.docx
eugeniadean34240
 
1_2 Introduction to Machine Learning.pdf
1_2 Introduction to Machine Learning.pdf1_2 Introduction to Machine Learning.pdf
1_2 Introduction to Machine Learning.pdf
RaviBhuva13
 
Boosting dl concept learners
Boosting dl concept learners Boosting dl concept learners
Boosting dl concept learners
Giuseppe Rizzo
 
15 ch ken black solution
15 ch ken black solution15 ch ken black solution
15 ch ken black solution
Krunal Shah
 
Andres hernandez ai_machine_learning_london_nov2017
Andres hernandez ai_machine_learning_london_nov2017Andres hernandez ai_machine_learning_london_nov2017
Andres hernandez ai_machine_learning_london_nov2017
Andres Hernandez
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
nextlib
 
PERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNING
PERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNINGPERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNING
PERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNING
abeeratariq20011
 
4.Support Vector Machines.ppt machine learning and development
4.Support Vector Machines.ppt machine learning and development4.Support Vector Machines.ppt machine learning and development
4.Support Vector Machines.ppt machine learning and development
PriyankaRamavath3
 
Computational Learning Theory
Computational Learning TheoryComputational Learning Theory
Computational Learning Theory
butest
 
ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING
ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MININGANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING
ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING
csandit
 
Analytical study of feature extraction techniques in opinion mining
Analytical study of feature extraction techniques in opinion miningAnalytical study of feature extraction techniques in opinion mining
Analytical study of feature extraction techniques in opinion mining
csandit
 
Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...
Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...
Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...
cscpconf
 
boosting algorithm
boosting algorithmboosting algorithm
boosting algorithm
Prithvi Paneru
 
Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)
VARUN KUMAR
 
13ClassifierPerformance.pdf
13ClassifierPerformance.pdf13ClassifierPerformance.pdf
13ClassifierPerformance.pdf
ssuserdce5c21
 
Understanding Blackbox Prediction via Influence Functions
Understanding Blackbox Prediction via Influence FunctionsUnderstanding Blackbox Prediction via Influence Functions
Understanding Blackbox Prediction via Influence Functions
SEMINARGROOT
 
14 ch ken black solution
14 ch ken black solution14 ch ken black solution
14 ch ken black solution
Krunal Shah
 
Introduction to Machine Learning Lectures
Introduction to Machine Learning LecturesIntroduction to Machine Learning Lectures
Introduction to Machine Learning Lectures
ssuserfece35
 
Machine learning in science and industry — day 1
Machine learning in science and industry — day 1Machine learning in science and industry — day 1
Machine learning in science and industry — day 1
arogozhnikov
 
26 Ch. 3 Organizing and Graphing DataAssignment 2ME.docx
26     Ch. 3 Organizing and Graphing DataAssignment 2ME.docx26     Ch. 3 Organizing and Graphing DataAssignment 2ME.docx
26 Ch. 3 Organizing and Graphing DataAssignment 2ME.docx
eugeniadean34240
 
1_2 Introduction to Machine Learning.pdf
1_2 Introduction to Machine Learning.pdf1_2 Introduction to Machine Learning.pdf
1_2 Introduction to Machine Learning.pdf
RaviBhuva13
 
Boosting dl concept learners
Boosting dl concept learners Boosting dl concept learners
Boosting dl concept learners
Giuseppe Rizzo
 
15 ch ken black solution
15 ch ken black solution15 ch ken black solution
15 ch ken black solution
Krunal Shah
 
Andres hernandez ai_machine_learning_london_nov2017
Andres hernandez ai_machine_learning_london_nov2017Andres hernandez ai_machine_learning_london_nov2017
Andres hernandez ai_machine_learning_london_nov2017
Andres Hernandez
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
nextlib
 
PERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNING
PERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNINGPERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNING
PERFORMANCE EVALUATION PARAMETERS FOR MACHINE LEARNING
abeeratariq20011
 
4.Support Vector Machines.ppt machine learning and development
4.Support Vector Machines.ppt machine learning and development4.Support Vector Machines.ppt machine learning and development
4.Support Vector Machines.ppt machine learning and development
PriyankaRamavath3
 
Computational Learning Theory
Computational Learning TheoryComputational Learning Theory
Computational Learning Theory
butest
 
ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING
ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MININGANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING
ANALYTICAL STUDY OF FEATURE EXTRACTION TECHNIQUES IN OPINION MINING
csandit
 
Analytical study of feature extraction techniques in opinion mining
Analytical study of feature extraction techniques in opinion miningAnalytical study of feature extraction techniques in opinion mining
Analytical study of feature extraction techniques in opinion mining
csandit
 
Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...
Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...
Radial Basis Function Neural Network (RBFNN), Induction Motor, Vector control...
cscpconf
 
Ad

More from VARUN KUMAR (20)

Distributed rc Model
Distributed rc ModelDistributed rc Model
Distributed rc Model
VARUN KUMAR
 
Electrical Wire Model
Electrical Wire ModelElectrical Wire Model
Electrical Wire Model
VARUN KUMAR
 
Interconnect Parameter in Digital VLSI Design
Interconnect Parameter in Digital VLSI DesignInterconnect Parameter in Digital VLSI Design
Interconnect Parameter in Digital VLSI Design
VARUN KUMAR
 
Introduction to Digital VLSI Design
Introduction to Digital VLSI DesignIntroduction to Digital VLSI Design
Introduction to Digital VLSI Design
VARUN KUMAR
 
Challenges of Massive MIMO System
Challenges of Massive MIMO SystemChallenges of Massive MIMO System
Challenges of Massive MIMO System
VARUN KUMAR
 
E-democracy or Digital Democracy
E-democracy or Digital DemocracyE-democracy or Digital Democracy
E-democracy or Digital Democracy
VARUN KUMAR
 
Ethics of Parasitic Computing
Ethics of Parasitic ComputingEthics of Parasitic Computing
Ethics of Parasitic Computing
VARUN KUMAR
 
Action Lines of Geneva Plan of Action
Action Lines of Geneva Plan of ActionAction Lines of Geneva Plan of Action
Action Lines of Geneva Plan of Action
VARUN KUMAR
 
Geneva Plan of Action
Geneva Plan of ActionGeneva Plan of Action
Geneva Plan of Action
VARUN KUMAR
 
Fair Use in the Electronic Age
Fair Use in the Electronic AgeFair Use in the Electronic Age
Fair Use in the Electronic Age
VARUN KUMAR
 
Software as a Property
Software as a PropertySoftware as a Property
Software as a Property
VARUN KUMAR
 
Orthogonal Polynomial
Orthogonal PolynomialOrthogonal Polynomial
Orthogonal Polynomial
VARUN KUMAR
 
Patent Protection
Patent ProtectionPatent Protection
Patent Protection
VARUN KUMAR
 
Copyright Vs Patent and Trade Secrecy Law
Copyright Vs Patent and Trade Secrecy LawCopyright Vs Patent and Trade Secrecy Law
Copyright Vs Patent and Trade Secrecy Law
VARUN KUMAR
 
Property Right and Software
Property Right and SoftwareProperty Right and Software
Property Right and Software
VARUN KUMAR
 
Investigating Data Trials
Investigating Data TrialsInvestigating Data Trials
Investigating Data Trials
VARUN KUMAR
 
Gaussian Numerical Integration
Gaussian Numerical IntegrationGaussian Numerical Integration
Gaussian Numerical Integration
VARUN KUMAR
 
Censorship and Controversy
Censorship and ControversyCensorship and Controversy
Censorship and Controversy
VARUN KUMAR
 
Romberg's Integration
Romberg's IntegrationRomberg's Integration
Romberg's Integration
VARUN KUMAR
 
Introduction to Censorship
Introduction to Censorship Introduction to Censorship
Introduction to Censorship
VARUN KUMAR
 
Distributed rc Model
Distributed rc ModelDistributed rc Model
Distributed rc Model
VARUN KUMAR
 
Electrical Wire Model
Electrical Wire ModelElectrical Wire Model
Electrical Wire Model
VARUN KUMAR
 
Interconnect Parameter in Digital VLSI Design
Interconnect Parameter in Digital VLSI DesignInterconnect Parameter in Digital VLSI Design
Interconnect Parameter in Digital VLSI Design
VARUN KUMAR
 
Introduction to Digital VLSI Design
Introduction to Digital VLSI DesignIntroduction to Digital VLSI Design
Introduction to Digital VLSI Design
VARUN KUMAR
 
Challenges of Massive MIMO System
Challenges of Massive MIMO SystemChallenges of Massive MIMO System
Challenges of Massive MIMO System
VARUN KUMAR
 
E-democracy or Digital Democracy
E-democracy or Digital DemocracyE-democracy or Digital Democracy
E-democracy or Digital Democracy
VARUN KUMAR
 
Ethics of Parasitic Computing
Ethics of Parasitic ComputingEthics of Parasitic Computing
Ethics of Parasitic Computing
VARUN KUMAR
 
Action Lines of Geneva Plan of Action
Action Lines of Geneva Plan of ActionAction Lines of Geneva Plan of Action
Action Lines of Geneva Plan of Action
VARUN KUMAR
 
Geneva Plan of Action
Geneva Plan of ActionGeneva Plan of Action
Geneva Plan of Action
VARUN KUMAR
 
Fair Use in the Electronic Age
Fair Use in the Electronic AgeFair Use in the Electronic Age
Fair Use in the Electronic Age
VARUN KUMAR
 
Software as a Property
Software as a PropertySoftware as a Property
Software as a Property
VARUN KUMAR
 
Orthogonal Polynomial
Orthogonal PolynomialOrthogonal Polynomial
Orthogonal Polynomial
VARUN KUMAR
 
Patent Protection
Patent ProtectionPatent Protection
Patent Protection
VARUN KUMAR
 
Copyright Vs Patent and Trade Secrecy Law
Copyright Vs Patent and Trade Secrecy LawCopyright Vs Patent and Trade Secrecy Law
Copyright Vs Patent and Trade Secrecy Law
VARUN KUMAR
 
Property Right and Software
Property Right and SoftwareProperty Right and Software
Property Right and Software
VARUN KUMAR
 
Investigating Data Trials
Investigating Data TrialsInvestigating Data Trials
Investigating Data Trials
VARUN KUMAR
 
Gaussian Numerical Integration
Gaussian Numerical IntegrationGaussian Numerical Integration
Gaussian Numerical Integration
VARUN KUMAR
 
Censorship and Controversy
Censorship and ControversyCensorship and Controversy
Censorship and Controversy
VARUN KUMAR
 
Romberg's Integration
Romberg's IntegrationRomberg's Integration
Romberg's Integration
VARUN KUMAR
 
Introduction to Censorship
Introduction to Censorship Introduction to Censorship
Introduction to Censorship
VARUN KUMAR
 
Ad

Recently uploaded (20)

Modeling the Influence of Environmental Factors on Concrete Evaporation Rate
Modeling the Influence of Environmental Factors on Concrete Evaporation RateModeling the Influence of Environmental Factors on Concrete Evaporation Rate
Modeling the Influence of Environmental Factors on Concrete Evaporation Rate
Journal of Soft Computing in Civil Engineering
 
Agents chapter of Artificial intelligence
Agents chapter of Artificial intelligenceAgents chapter of Artificial intelligence
Agents chapter of Artificial intelligence
DebdeepMukherjee9
 
Nanometer Metal-Organic-Framework Literature Comparison
Nanometer Metal-Organic-Framework  Literature ComparisonNanometer Metal-Organic-Framework  Literature Comparison
Nanometer Metal-Organic-Framework Literature Comparison
Chris Harding
 
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
Guru Nanak Technical Institutions
 
Using the Artificial Neural Network to Predict the Axial Strength and Strain ...
Using the Artificial Neural Network to Predict the Axial Strength and Strain ...Using the Artificial Neural Network to Predict the Axial Strength and Strain ...
Using the Artificial Neural Network to Predict the Axial Strength and Strain ...
Journal of Soft Computing in Civil Engineering
 
Slide share PPT of SOx control technologies.pptx
Slide share PPT of SOx control technologies.pptxSlide share PPT of SOx control technologies.pptx
Slide share PPT of SOx control technologies.pptx
vvsasane
 
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdfDavid Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry
 
DED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedungDED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedung
nabilarizqifadhilah1
 
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
PawachMetharattanara
 
Applications of Centroid in Structural Engineering
Applications of Centroid in Structural EngineeringApplications of Centroid in Structural Engineering
Applications of Centroid in Structural Engineering
suvrojyotihalder2006
 
Design Optimization of Reinforced Concrete Waffle Slab Using Genetic Algorithm
Design Optimization of Reinforced Concrete Waffle Slab Using Genetic AlgorithmDesign Optimization of Reinforced Concrete Waffle Slab Using Genetic Algorithm
Design Optimization of Reinforced Concrete Waffle Slab Using Genetic Algorithm
Journal of Soft Computing in Civil Engineering
 
Machine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATIONMachine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATION
DarrinBright1
 
2.3 Genetically Modified Organisms (1).ppt
2.3 Genetically Modified Organisms (1).ppt2.3 Genetically Modified Organisms (1).ppt
2.3 Genetically Modified Organisms (1).ppt
rakshaiya16
 
Modelling of Concrete Compressive Strength Admixed with GGBFS Using Gene Expr...
Modelling of Concrete Compressive Strength Admixed with GGBFS Using Gene Expr...Modelling of Concrete Compressive Strength Admixed with GGBFS Using Gene Expr...
Modelling of Concrete Compressive Strength Admixed with GGBFS Using Gene Expr...
Journal of Soft Computing in Civil Engineering
 
introduction technology technology tec.pptx
introduction technology technology tec.pptxintroduction technology technology tec.pptx
introduction technology technology tec.pptx
Iftikhar70
 
Mode-Wise Corridor Level Travel-Time Estimation Using Machine Learning Models
Mode-Wise Corridor Level Travel-Time Estimation Using Machine Learning ModelsMode-Wise Corridor Level Travel-Time Estimation Using Machine Learning Models
Mode-Wise Corridor Level Travel-Time Estimation Using Machine Learning Models
Journal of Soft Computing in Civil Engineering
 
Little Known Ways To 3 Best sites to Buy Linkedin Accounts.pdf
Little Known Ways To 3 Best sites to Buy Linkedin Accounts.pdfLittle Known Ways To 3 Best sites to Buy Linkedin Accounts.pdf
Little Known Ways To 3 Best sites to Buy Linkedin Accounts.pdf
gori42199
 
Automatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and BeyondAutomatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and Beyond
NU_I_TODALAB
 
Construction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil EngineeringConstruction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil Engineering
Lavish Kashyap
 
ML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdf
ML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdfML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdf
ML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdf
rameshwarchintamani
 
Agents chapter of Artificial intelligence
Agents chapter of Artificial intelligenceAgents chapter of Artificial intelligence
Agents chapter of Artificial intelligence
DebdeepMukherjee9
 
Nanometer Metal-Organic-Framework Literature Comparison
Nanometer Metal-Organic-Framework  Literature ComparisonNanometer Metal-Organic-Framework  Literature Comparison
Nanometer Metal-Organic-Framework Literature Comparison
Chris Harding
 
Slide share PPT of SOx control technologies.pptx
Slide share PPT of SOx control technologies.pptxSlide share PPT of SOx control technologies.pptx
Slide share PPT of SOx control technologies.pptx
vvsasane
 
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdfDavid Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry
 
DED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedungDED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedung
nabilarizqifadhilah1
 
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
PawachMetharattanara
 
Applications of Centroid in Structural Engineering
Applications of Centroid in Structural EngineeringApplications of Centroid in Structural Engineering
Applications of Centroid in Structural Engineering
suvrojyotihalder2006
 
Machine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATIONMachine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATION
DarrinBright1
 
2.3 Genetically Modified Organisms (1).ppt
2.3 Genetically Modified Organisms (1).ppt2.3 Genetically Modified Organisms (1).ppt
2.3 Genetically Modified Organisms (1).ppt
rakshaiya16
 
introduction technology technology tec.pptx
introduction technology technology tec.pptxintroduction technology technology tec.pptx
introduction technology technology tec.pptx
Iftikhar70
 
Little Known Ways To 3 Best sites to Buy Linkedin Accounts.pdf
Little Known Ways To 3 Best sites to Buy Linkedin Accounts.pdfLittle Known Ways To 3 Best sites to Buy Linkedin Accounts.pdf
Little Known Ways To 3 Best sites to Buy Linkedin Accounts.pdf
gori42199
 
Automatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and BeyondAutomatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and Beyond
NU_I_TODALAB
 
Construction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil EngineeringConstruction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil Engineering
Lavish Kashyap
 
ML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdf
ML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdfML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdf
ML_Unit_VI_DEEP LEARNING_Introduction to ANN.pdf
rameshwarchintamani
 

Vc dimension in Machine Learning

  • 1. VC Dimension in Machine Learning Dr. Varun Kumar Dr. Varun Kumar Lecture 18 1 / 10
  • 2. Outlines 1 General Classification Problem 2 Usage of VC dimension in ML 3 Introduction to Vapnik-Chervonenkis (VC) Dimension 4 How to Determine VC Dimension for a Given Classifier or Hypothesis? 5 References Dr. Varun Kumar Lecture 18 2 / 10
  • 3. General classification problem 1 Always look for test error along with the training error. 2 Improving on training error does not improve the test error. 3 Increase in machine capacity may give the poor performance. Is there any equation that relates the training and test error ? Dr. Varun Kumar Lecture 18 3 / 10
  • 4. Usage of VC dimension in ML Model complexity determines the performance/cost on both the training and test sets. P Test error ≤ Training error + r h(log(2N/h) + 1) − log η/4 N = 1 − η Note: Above expression shows the upper bound of test error with probability 1 − η. h→ VC dimension h measure the power h does not depend on the choice of training set N → Total number of training sample For reducing the residual, h → low or N → high Test error ≤ Training error + Penalty(Complexity) . Dr. Varun Kumar Lecture 18 4 / 10
  • 5. Continued– ⇒ Let us our training data are iid from some distribution fX (x). ⇒ Types of risk (i) Risk R(θ)→ Long term observation→ Test observation R(θ) = Test error = E[δ(c 6= ĉ(x; θ))] (ii) Empirical risk Remp (θ)→ Finite sample observation→ Training observation Remp (θ) = Training error = 1 m X i [δ(c(i) 6= ĉ(i) (x; θ))] Dr. Varun Kumar Lecture 18 5 / 10
  • 6. Introduction to Vapnik-Chervonenkis (VC) Dimension Key features: ⇒ VC dimension is a measure of the capacity (complexity, expressive power, richness, or flexibility) of a set of functions. ⇒ It learns by a statistical binary classification algorithm. ⇒ It is defined as the cardinality of the largest set of points that the algorithm can shatter. Cardinality refers to the size of set. Ex- A = {1, 4, 6}, cardinality |A| = 3 ⇒ The capacity of a classification model is related to how complicated it can be.→ Overfitting VC dimension of a set-family Let H be a set family (a set of sets) and C a set. H ∩ C := {h ∩ C | h ∈ H}. Dr. Varun Kumar Lecture 18 6 / 10
  • 7. Relationship between risk and model complexity Dr. Varun Kumar Lecture 18 7 / 10
  • 8. How to determine VC dimension for a given classifier or hypothesis? 1 General point setting: Statement: In a n−dimensional feature space a set of m points (m n) is in general position if and only if no subset of (m + 1) points lie on the (n − 1) dimensional hyperplane. Dr. Varun Kumar Lecture 18 8 / 10
  • 9. 2 Shattering: Statement: A hypothesis H shatter m points in n− dimensional space if all possible combinations of m points in n− dimensional space are correctly classified. Dr. Varun Kumar Lecture 18 9 / 10
  • 10. References E. Alpaydin, Introduction to machine learning. MIT press, 2020. T. M. Mitchell, The discipline of machine learning. Carnegie Mellon University, School of Computer Science, Machine Learning , 2006, vol. 9. J. Grus, Data science from scratch: first principles with python. O’Reilly Media, 2019. Dr. Varun Kumar Lecture 18 10 / 10
  翻译: