Lecture 1 graphical models
Download as pptx, pdf3 likes4,771 views
This document provides an overview of probabilistic graphical models. It discusses two types of probabilistic graphical models - Bayesian networks and Markov networks. Bayesian networks use directed graphs to represent conditional independence relationships between random variables. Markov networks use undirected graphs for the same purpose. The document outlines topics like representation, examples including naive Bayes classifiers and the Ising model, and inference and learning algorithms for probabilistic graphical models.
More Related Content
What's hot (20)
Similar to Lecture 1 graphical models (16)
Recently uploaded (20)
Lecture 1 graphical models
- 1. Machine Learning Lecture 1: Probabilistic Graphical Models Phạm Duy Tùng Email: duytung88@gmail.com 9/9/2012 9/17/2012 Some slides copied from Pattern Recognition and Machine Learning (Bishop 2006) 1
- 2. Introduction • Probabilistic graphical models • A visual presentation of probability distributions, using diagrams, called PGMs • Two types of PGM • Bayesian Network (Directed Graphical Model) • Markov Network (Undirected Graphical Model) 9/17/2012 2
- 3. Introduction • Offering several useful properties: • They provide a simply way to visualize the structure of probabilistic models and motivate new models. • Insights into the properties of the models, including the conditional independence properties, can be obtain by inspection of the graph. • Graph based algorithms for calculation and computation 9/17/2012 3
- 4. Topics • Introduction • Representation • Bayesian Networks (Directed Graphical Model) • Markov Networks (Undirected Graphical Model) • Converting Bayesian Networks to Markov Networks • Directed vs. Undirected Graphs • Some examples • Naïve Bayes Classifier • Ising Model • Inference • Learning 9/17/2012 4
- 5. Representation • Random variables -> Nodes • In Conditional Independences -> Edges (Directed or Undirected) Probability Graphical Models Models 9/17/2012 5
- 6. Topics • Introduction • Representation • Bayesian Networks (Directed Graphical Model) • Markov Networks (Undirected Graphical Model) • Converting Bayesian Networks to Markov Networks • Directed vs. Undirected Graphs • Some examples • Naïve Bayes Classifier • Ising Model • Inference • Learning 9/17/2012 6
- 7. Representation->Bayesian Network • Using Directed Acyclic Graph (DAG) • Joint distribution factorizes according to graph 9/17/2012 7
- 9. Representation->Bayesian Network • Conditional independence: Or equivalently: Denoted by: 9/17/2012 9
- 10. Representation->Bayesian Network • Conditional independence: Example 1 9/17/2012 10
- 11. Representation->Bayesian Network • Conditional independence: Example 1 9/17/2012 11
- 12. Representation->Bayesian Network • Conditional independence: Example 2 9/17/2012 12
- 13. Representation->Bayesian Network • Conditional independence: Example 2 9/17/2012 13
- 14. Representation->Bayesian Network • Conditional independence: Example 3 9/17/2012 14
- 15. Representation->Bayesian Network • Conditional independence: Example 3 9/17/2012 15
- 16. Representation->Bayesian Network • D-separation • A, B, and C are non-intersecting subsets of nodes in a directed graph. • A path from A to B is blocked if it contains a node such that either a)the arrows on the path meet either head-to-tail or tail-to-tail at the node, and the node is in the set C, or b)the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, are in the set C. • If all paths from A to B are blocked, A is said to be d-separated from B by C. • If A is d-separated from B by C, the joint distribution over all variables in the graph satisfies . 9/17/2012 16
- 17. Representation->Bayesian Network • D-separation: Example 9/17/2012 17
- 18. Representation->Bayesian Network • Markov Blanket Factors independent of xi cancel between numerator and denominator. 9/17/2012 18
- 19. Example • Mixture of Gaussians 9/17/2012 19
- 20. Home Work • LDA model (David M.Blei) 9/17/2012 20
- 21. Topics • Introduction • Representation • Bayesian Networks (Directed Graphical Model) • Markov Networks (Undirected Graphical Model) • Converting Bayesian Networks to Markov Networks • Directed vs. Undirected Graphs • Some examples • Naïve Bayes Classifier • Ising Model • Inference • Learning 9/17/2012 21
- 22. Representation->Markov Network • Many phenomenon in real life, we can not determine exactly the directionality to the interaction between random variables. • We use Markov Network to modeling these phenomenon instead of Bayesian Network. 9/17/2012 22
- 23. Representation->Markov Network • Conditional independence Markov Blanket 9/17/2012 23
- 24. Representation->Markov Network • Factorization properties 9/17/2012 24
- 25. Representation->Markov Network • Clique Clique Maximal Clique 9/17/2012 25
- 26. Representation->Markov Network • where is the potential over clique C and is the normalization coefficient; note: M K-state variables KM terms in Z. • Energies and the Boltzmann distribution 9/17/2012 26
- 27. Representation -> Converting Bayesian Networks to Markov Networks 9/17/2012 27
- 28. Representation -> Converting Bayesian Networks to Markov Networks • Additional links 9/17/2012 28
- 29. Directed vs. Undirected Graphs 9/17/2012 29
- 30. Directed vs. Undirected Graphs
- 31. Topics • Introduction • Representation • Bayesian Networks (Directed Graphical Model) • Markov Networks (Undirected Graphical Model) • Converting Bayesian Networks to Markov Networks • Directed vs. Undirected Graphs • Some examples • Naïve Bayes Classifier • Ising Model • Inference • Learning 9/17/2012 31
- 32. Naïve Bayes Classifier • Predicting 9/17/2012 32
- 33. Naïve Bayes Classifier • How to estimate and ? • Solution: We can separately estimate two parts of the model using Maximum Likelihood 9/17/2012 33
- 44. Ising Model and Image de-noising • Markov Random Field 9/17/2012 44
- 45. Ising Model and Image de-noising • An example of Ising Model used in Image processing. Noisy Image Original Image 9/17/2012 45
- 46. Ising Model and Image de-noising 9/17/2012 46
- 47. Ising Model and Image de-noising • All of our assumptions about images are encoded as follows: 9/17/2012 47
- 48. Ising Model and Image de-noising 9/17/2012 48
- 49. Ising Model and Image de-noising 9/17/2012 49
- 50. Ising Model and Image de-noising 9/17/2012 50
- 51. Ising Model and Image de-noising • Results Noisy Image Restored Image (ICM) 9/17/2012 51
- 52. Ising Model and Image de-noising • Comparing two optimizing algorithms: Graph cuts vs ICM Restored Image (ICM) Restored Image (Graph cuts) 9/17/2012 52
- 53. Ising Model and Image de-noising • Home works – Design and implement an image segmentation algorithms using Ising model. 9/17/2012 53
- 54. Topics • Introduction • Representation • Bayesian Networks (Directed Graphical Model) • Markov Networks (Undirected Graphical Model) • Converting Bayesian Networks to Markov Networks • Directed vs. Undirected Graphs • Some examples • Naïve Bayes Classifier • Ising Model • Inference • Learning 9/17/2012 54
- 55. Inference 9/17/2012 55
- 56. Learning 9/17/2012 56