CSIT 5220
Page 1
Recap of L08: Structure Learning
CSIT 5220
Page 2
Review of L08: Structure Learning

Score-based approach

Constraint-based approach
CSIT 5220
Summary of Maximized Likelihood
Structure
Maximized Loglikelhood
-11.26
-10
-8
Maximized likelihood does not work

Lead to overfitting, i.e., very complexity model structure
Page 3
CSIT 5220
Page 4
CSIT 5220
Structure
Page 5
Example (Cont’d)
loglikehihood
Complexity
-11.26
3
-10
4
-8
7
BIC
-14.26
-14
-15
CSIT 5220
Page 6
Model Optimization: Hill Climbing
CSIT 5220
Page 7
CSIT 5220

Page 8
Hint:

Start with complete graph

Delete the edge between pairs of variables that are independence under
some condition
CSIT 5220
Page 9
CSIT 5220
After Rule 1 Has Been Exhausted….
CSIT 5220
Page 11
The PC Algorithm

Start with the complete graph

For each link A –B




Test whether A and B independent given 0 neighbor of A

If yes, remove the link
For each remaining link A – B

Test whether A and B independent given 1 neighbor of A

If yes, remove the link
For each remaining link A – B

Test whether A and B independent given 2 neighbors of A

If yes, remove the link
…..