COS402– Machine
Learningand
ArtificialIntelligence
Fall2016
Lecture12:KnowledgeRepresentationand
ReasoningPart1:Logic
SanjeevArora
EladHazan
(BorrowsfromslidesofPercyLiang,StanfordU.)
Thusfarinthecourse
• Formalizationoflearningfromdata(statisticallearningtheory)
• Languagemodelsandlanguagesemantics.(examplesofunsupervisedlearning)
•
Recommendersystems.
Today:Knowledgerepresentationandreasoningusinglogic
Reminder:In-classmidtermthisThurs.75min;closedbook(arriveontime!)
(Studyguidepostedonpiazza.)
Alsobasisofdigitalcircuitsincomputerchips
EE206/COS306
RoleoflogicinAI
• For2000years,peopletriedtocodify“humanreasoning”andcameupwithlogic.
• MostAIworkuntil1980s:Buildmachinesthatrepresentknowledgeand
doreasoningvialogic.“Rulebasedreasoning.”
• ”Learningfromdata”ispopulartoday,butlacksaspectsthatweretrivialin
thepre-1980ssystems(e.g.allowhumanprogrammertoeasilycommunicate
his/herknowledgetothesystem).“Howdoyouteachadeepnettomultiply
twonumbers?”
• Logicalreasoningnowseemspoisedforacomeback.
Goalsoflogic
• Represent knowledgeabouttheworld.
• Reason withthatknowledge.
Naturallanguage?
• Adime isbetterthananickel.
• Anickel isbetterthanapenny.
• Apennyisbetterthananothing.
Knowledge
• Therefore,adime isbetterthanapenny.
Reasoning
• Nothing isbetterthanworldpeace.
• Therefore,apenny isbetterthanworldpeace.
Naturallanguageistricky!
Useoflogicremovesambiguity(similartocomputerlanguages);
butalsomakessystemlessflexible.(Willstudymoreflexibleversionslater.)
Componentsofanylogicalsystem
• Syntax
• Semantics.
• Reasoning
Differentsyntax,samesemantics
2+3⇔ 3+2
Samesyntax,differentsemantics
3/2inPython2.7vs3/2inPython3.
PropositionalLogic(akaBooleanLogic;rememberCOS126!)
Syntax:
(A _ ¬B) ^ (¬A _ B)
⋁ and⋀ and⟷ aresymmetric,like“+”and“times”inarithmetic.
Syntaxprovidessymbols.
No“meaning”yet(semantics)!
Semanticsprovidedbya“Model”
(unrelatedto“model”usedinmachinelearning!)
Forpropositionallogic,amodelissimplyan
assignmenttoallvariables.
(eachvariableassigned0(false)or1(true),
not both)
Sanitycheck:Whatis#of
possiblemodels ifthereare
3variables?Howabout
nvariables?
Interpretationfunction
I(f,w):Givenformulafandmodelw,assignsexactly
oneof1(True)or0(False)tof.
•
•
•
•
•
Trueiff fisfalse
Trueiff bothfandgaretrue
Trueiff atleastoneoff,gistrue
Falseiff fistrueandgisfalse.
Trueiff fandghavethesamevalue(trueorfalse)
Definition:
M(f)=Setofmodelswforwhich I(f,w)=True
Formulafcompactly represents
M(f)(“Setofpossibleworldswhere
fistrue.”)
Tautology: FormulafsuchthatM(f)=All
possiblemodels.(“Trueinallpossibleworlds”)
• Example:Forf=A⋀ B
M(f)={A=1,B=1}.
• Forf=A⟷ B
M(f)={A=1,B=1};{A=0;B=0}
Example:A⋁ ¬A.
(TruewhetherA=0orA=1!)
Contradiction: FormulafsuchthatM(f)=Empty
set.(”Falseinallpossibleworlds.”)
Example:A⋀ ¬A.
(FalsewhetherA=0orA=1!)
Knowledgerepresentationvialogic
Knowledgebase:Setofformulae{f1,f2,…, fn}
M(KB)=Allpossiblemodelsforf1⋀ f2⋀… ⋀ fn
Formulae=“knownfacts”
Models=allpossible“worlds”where
allthesefactshold
(AddingmorefactstoKBcanonly
shrink setofpossibleworlds.)
Example:Variables:R,S,C(“Rainy”,“Sunny,”“Cloudy”)
KB:R⋁ S⋁ C;
R→C⋀ ¬ S;
C⟷ ¬S
(”ItiseitherRainyorSunnyorCloudy.”)
(“IfitisRainythenitisCloudyandnotSunny.”)
(”IfitisCloudythenitisnotSunny,andviceversa”)
ModelsforKB:{R=1,S=0,C=1};{R=0,C=1,S=0};{R=0,C=0,S=1}.
Satisfiability
Defn:Knowledge-baseKBissatisfiable if
M(KB)≠∅
(i.e.thereissomeassignmenttovariables
thatmakesallformulaeinKBevaluatetoTrue)
Defn:KBentails formulaf(denotedKB⊨ f)if
M(KB∪{f})=M(KB).
(ineveryworldwhereKBistrue,fisalsotrue)
Sanitycheck:KBentailsfiff itcontradicts¬f.
Defn:KBcontradicts formulafif
KB∪{f}isnotsatisfiable
Defn:KBisconsistentwith
formulafif
M(KB∪{f})isnon-empty
(thereisaworldinwhichKBis
trueandfisalsotrue)
Anexample
Example:Variables:R,S,C(“Rainy”,“Sunny,”“Cloudy”)
KB: R⋁ S⋁ C;
R→C⋀ ¬S;
C⟷ ¬S
DoesKB⊨ S⋁ C?
ModelsforKB:{R=1,S=0,C=1};{R=0,C=1,S=0};{R=0,C=0,S=1}.
S⋁Cistrueinallthesemodels✔ Anexample
Example:Variables:R,S,C(“Rainy”,“Sunny,”“Cloudy”)
KB: R⋁ S⋁ C;
R→C⋀ ¬S;
C⟷ ¬S
Examples:R →U;S→¬U;
Addavariable:U(“Carryan
umbrella”).Whatcommon-sense
“facts”canweaddaboutUtothe
aboveKB?
AIsystemstill1980susedsuchreasoning;
”facts”wereaddedbyprogrammersbased
uponintrospection.
Decision-makingatrun-time=
whichformulaeare
entailed/contradicted/consistent
Recapoflogicsofar
• Defn offormulae.
• KB=Listofformulae.(“Factsabouttheworld”)
• KBcanentail orcontradict anotherformula,orbeconsistentwithit.
• Todecidewhichofthethreepossibilitiesofprev.lineholds,drawup
listofallpossiblemodels.(“Truthtablemethod.”)
Truthtablemethod(tocheckifKBhasanymodel)
• Ifnvariables,cantake2n time.(infeasibleforevenn=100)
Anyfasteralgorithm?
• Polynomialtimealgorithmè P=NP(Famousopenproblem)
• Inpracticetherearereasonablealgorithmsthatuseresolution and
otherrelatedreasoningmethods.
ResolutionproceduretodecidesatisfiabilityofaKB
(simplestversion;[Davis-Putnam,1950s])
KBconsistsonlyofformulaethatareclauses (ie ⋁ ofvariablesornegatedvariables).
(Withsomework,canconverteveryKBtothisform.)
Warmup:Whatcanweconcludeunderfoll.conditions?
KBhassingletonclauses(A),(¬A).
HASNOMODEL(UNSATISFIABLE)!
KBcontainsclausepairsofform(A ⋁ B1 ⋁ … ⋁ Bn)and(¬A⋁ C1 ⋁ … ⋁ Cm)
EverymodelforKBmustmake(B1 ⋁ … ⋁ Bn ⋁ C1 ⋁ … ⋁ Cm)TRUE
ResolutionproceduretodecidesatisfiabilityofaKB
(simplestversion;[Davis-Putnam,1950s])
KBconsistsonlyofformulaethatareclauses (ie ⋁ ofvariablesornegatedvariables).
(Withsomework,canconverteveryKBtothisform.)
Claim(won’tprove):Finishesinfinitetime
While KBnonemptydo
foreveryKBandprintscorrectanswer.
{
If KBcontainsclausepairsofform(A),(¬A)
Print (“NoModel.”)andSTOP. /*WHY?*/
If KBcontainsclausepairsofform(A ⋁ B1 ⋁ … ⋁ Bn)and(¬A⋁ C1 ⋁ … ⋁ Cm)
Add(B1 ⋁ … ⋁ Bn ⋁ C1 ⋁ … ⋁ Cm)toKB./*WHY?*/
else
Print(“Modelexists”)andSTOP. /*WHY??*/
}
Goodluckwithmidterm,
Andhaveagoodfallbreak!