DATA130008 Introduction to Artificial Intelligence
复旦大学大数据学院
School of Data Science, Fudan University
魏忠钰
Bayes’ Nets: Independence
May3rd,2017
Bayes’Nets
复旦大学大数据学院
School of Data Science, Fudan University
§ ABayes’netisan efficientencoding ofaprobabilistic
modelofadomain
§ Questionswecanask:
§ Inference:givenafixedBN,whatisP(X|e)?
§ Representation:givenaBNgraph,whatkindsofdistributions
canitencode?
§ Modeling:whatBNismostappropriateforagivendomain?
ConditionalIndependence
复旦大学大数据学院
School of Data Science, Fudan University
§ XandYareindependentif
§ XandYareconditionallyindependent givenZ
§ (Conditional)independenceisapropertyofadistribution
§ Example:
BayesNets:Assumptions
复旦大学大数据学院
School of Data Science, Fudan University
§ AssumptionswearerequiredtomaketodefinetheBayesnet
whengiventhegraph:
P (xi |x1 · · · xi
1)
= P (xi |parents(Xi ))
§ Beyondabove“chainruleà Bayesnet” conditional
independenceassumptions
§ Oftenadditionalconditionalindependences
§ Theycanbereadoffthegraph
§ Importantformodeling:understandassumptionsmadewhen
choosingaBayesnetgraph
Example
复旦大学大数据学院
School of Data Science, Fudan University
X
Y
Z
W
§ Conditionalindependenceassumptionsdirectlyfrom
simplificationsinchainrule:
§ Additionalimpliedconditionalindependenceassumptions?
IndependenceinaBN
复旦大学大数据学院
School of Data Science, Fudan University
§ ImportantquestionaboutaBN:
§ Aretwonodesindependentgivencertainevidence?
§ Ifyes,canproveusingalgebra(tediousingeneral)
§ Ifno,canprovewithacounterexample
§ Example:
X
Y
Z
§ Question:areXandZnecessarilyindependent?
§ Answer:no.Example:lowpressurecausesrain,whichcausestraffic.
§ XcaninfluenceZ,ZcaninfluenceX(viaY)
§ Theycouldbeindependent:how?
D-separation: Outline
复旦大学大数据学院
School of Data Science, Fudan University
D-separation: Outline
复旦大学大数据学院
School of Data Science, Fudan University
§ Studyindependencepropertiesfortriples
§ Analyzecomplexcasesintermsofmembertriples
§ D-separation:acondition/algorithmforanswering
suchqueries
CausalChains
复旦大学大数据学院
School of Data Science, Fudan University
§ Thisconfigurationisa”causalchain” § GuaranteedXindependentofZ? No!
§ OneexamplesetofCPTsforwhichXisnot
independentofZissufficienttoshowthis
independenceisnotguaranteed.
§ Example:
X:LowpressureY:Rain
Z:Traffic
§ Lowpressurecausesraincausestraffic,
highpressurecausesnoraincausesno
traffic
§ Innumbers:
P(+y|+x)=1,P(-y|- x)=1,
P(+z|+y)=1,P(-z|-y)=1
CausalChains
复旦大学大数据学院
School of Data Science, Fudan University
§ Thisconfigurationisa“causalchain”
X:LowpressureY:Rain
§ GuaranteedGivenY,Xindependentof
Z?
Z:Traffic
Yes!
§ Evidencealongthechain
“blocks” theinfluence
CommonCause
复旦大学大数据学院
School of Data Science, Fudan University
§ GuaranteedXindependentofZ? No!
Y:Project
due
§ OneexamplesetofCPTsforwhichXis
notindependentofZissufficientto
showthisindependenceisnot
guaranteed.
§ Example:
§ ProjectduecausesbothEmail box
busy andlibrary full
§ Innumbers
P(+x|+y)=1,P(-x|-y)=1,
P(+z|+y)=1,P(-z|-y)=1
X:Email Box
busy
Z:Library full
CommonCause
复旦大学大数据学院
School of Data Science, Fudan University
§ GuaranteedX andZ independentgivenY?
Yes!
Observingthecauseblocksinfluence
betweeneffects.
Y:Project
due
X:Email Box
busy
Z:Library full
CommonEffect
复旦大学大数据学院
School of Data Science, Fudan University
§ AreXandYindependent?
§ Yes:theballgameandtheraincausetraffic,but
theyarenotcorrelated
X:Raining
§ Stillneedtoprovetheymustbe
§ AreXandYindependentgivenZ?
§ No:seeingtrafficputstherainandthe
ballgameincompetitionasexplanation.
§ Thisisbackwardsfromtheothercases
§ Observinganeffectactivates influence
betweenpossiblecauses.
Z:Traffic
Y:Ballgame
The General Case
复旦大学大数据学院
School of Data Science, Fudan University
The General Case
复旦大学大数据学院
School of Data Science, Fudan University
§ Generalquestion:inagivenBN,aretwovariablesindependent
(givenevidence)?
§ Solution:analyzethegraph
§ Anycomplexexamplecanbebroken
intorepetitionsofthethreecanonicalcases
Reachability
复旦大学大数据学院
School of Data Science, Fudan University
§ Recipe:shadeevidencenodes,lookfor
pathsintheresultinggraph
L
§ Attempt1:iftwonodesareconnected
byanundirectedpathnotblockedbya
shadednode,theyareconditionally
independent
R
§ Almostworks,butnotquite
§ Wheredoesitbreak?
§ Answer:thev-structureatTdoesn’tcount
asalinkinapathunless“active”
D
B
T
Active/InactivePaths
§ Question:AreXandYconditionallyindependentgiven
evidencevariables{Z}?
复旦大学大数据学院
School of Data Science, Fudan University
InactiveTriples
§ Yes,ifXandY“d-separated” byZ
§ Considerall(undirected)pathsfromXtoY
§ Noactivepaths=independence!
§ Apathisactiveifevery tripleisactive:
§ CausalchainA® B® CwhereBisunobserved(eitherdirection)
§ CommoncauseA¬ B® CwhereBisunobserved
§ Commoneffect(akav-structure)
A® B¬ CwhereBoroneofitsdescendents isobserved
§ Allittakestoblockapathisasingleinactivesegment
ActiveTriples
D-Separation
§ Query:
复旦大学大数据学院
School of Data Science, Fudan University
Xi
Xj |{Xk1 , ..., Xkn }
?
§ Checkall(undirected!)pathsbetweenand
§ Ifoneormoreactive,thenindependencenotguaranteed
Xi
Xj |{Xk1 , ..., Xkn }
§ Otherwise(i.e.ifallpathsareinactive),
thenindependenceisguaranteed
Xi
Xj |{Xk1 , ..., Xkn }
Example
复旦大学大数据学院
School of Data Science, Fudan University
R
B
T
T’
Example
复旦大学大数据学院
School of Data Science, Fudan University
L
R
D
B
T
T’
Example
复旦大学大数据学院
School of Data Science, Fudan University
§ Variables:
§ R:Raining
§ T:Traffic
§ D:Roofdrips
§ S:I’msad
R
T
D
§ Questions:
S
Structure Implications
复旦大学大数据学院
School of Data Science, Fudan University
§ GivenaBayesnetstructure,canrund-separationalgorithm
tobuildacompletelistofconditionalindependencesthat
arenecessarilytrueoftheform
Xi
Xj |{Xk1 , ..., Xkn }
§ Thislistdeterminesthesetofprobabilitydistributionsthat
canberepresented
ComputingAllIndependences
复旦大学大数据学院
School of Data Science, Fudan University
Y
X
Z
Y
X
Z
X
Z
Y
Y
X
Z
TopologyLimitsDistributions
§ GivensomegraphtopologyG,
onlycertainjointdistributions
canbeencoded
§ Thegraphstructure
guaranteescertain
(conditional)independences
复旦大学大数据学院
School of Data Science, Fudan University
{X
X
Y, X
Z, Y
Z | Y, X
{X
Z,
Y | Z, Y
Z | X}
Y
X
Z | Y}
Y
X
Z
Z
Y
X
Z
Y
§ (Theremightbemore
independence)
X
§ Addingarcsincreasestheset
ofdistributions,buthas
severalcosts
Z
{}
§ Fullconditioningcanencode
anydistribution
Y
Y
X
Z
X
Z
X
Y
Y
X
Y
Z
X
Z
Y
Z
X
Z
BayesNetsRepresentationSummary
复旦大学大数据学院
School of Data Science, Fudan University
§ Bayesnetscompactlyencodejointdistributions
§ Guaranteedindependenciesofdistributionscanbe
deducedfromBNgraphstructure
§ D-separationgivespreciseconditionalindependence
guaranteesfromgraphalone
§ ABayes’net’sjointdistributionmayhavefurther
(conditional)independencethatisnotdetectableuntil
youinspectitsspecificdistribution
Bayes’ Nets
复旦大学大数据学院
School of Data Science, Fudan University
§ Representation
§ ConditionalIndependences
§ ProbabilisticInference
§ Enumeration(exact,exponentialcomplexity)
§ Variableelimination(exact,worst-case exponential
complexity,oftenbetter)
§ ProbabilisticinferenceisNP-complete
§ Sampling(approximate)
§ LearningBayes’ NetsfromData