Intelligent
Agents
Chapter2
Howdoyoudesignanintelligentagent?
• Intelligentagentsperceiveenvironmentviasensorsand
actra/onallyonthemwiththeireffectors
• Discreteagentsreceiveperceptsoneata/me,andmap
themtoasequenceofdiscreteac:ons
• Generalproper/es
– Reac/vetotheenvironment
– Pro-ac/veorgoal-directed
– Interactswithother
agentsthrough
communica/onor
viatheenvironment
– Autonomous
sensors/perceptsandeffectors/ac:ons?
Humanshave
–  Sensors:Eyes(vision),ears(hearing),skin(touch),tongue
(gusta/on),nose(olfac/on),neuromuscularsystem
(propriocep/on)
–  Percepts:
•  Atthelowestlevel:electricalsignalsfromthesesensors
•  AEerpreprocessing:objectsinthevisualfield(loca/on,
textures,colors,…),auditorystreams(pitch,loudness,
direc/on),…
–  Effectors:limbs,digits,eyes,tongue,…
–  Ac:ons:liEafinger,turnleE,walk,run,carryanobject,…
•  ThePoint:perceptsandac/onsneedtobecarefully
defined,possiblyatdifferentlevelsofabstrac/on
Example:autonomoustaxi
•  Percepts:Video,sonar,speedometer,
odometer,enginesensors,keyboard
input,microphone,GPS,…
•  Ac:ons:Steer,accelerate,brake,horn,speak,…
•  Goals:Maintainsafety,reachdes/na/on,maximize
profits(fuel,/rewear),obeylaws,provide
passengercomfort,…
•  Environment:U.S.urbanstreets,freeways,traffic,
pedestrians,weather,customers,…
•  Differentaspectsofdrivingmayrequiredifferent
typesofagentprograms!
Ra:onality
•  Idealra:onalagentsshould,foreachpercept
sequence,acttomaximizeexpectedperformance
measurebasedon
(1)theperceptsequence,and
(2)itsbuilt-inandacquiredknowledge
•  Ra/onalityincludesinforma/ongathering--Ifyoudon’t
knowsomething,findout!
•  Ra/onality→Needaperformancemeasuretosayhow
wellataskhasbeenachieved
•  Typesofperformancemeasures:falsealarm(false
posi/ve)andfalsedismissal(falsenega/ve)rates,
speed,resourcesrequired,effectonenvironment,etc.
Autonomy
•  Asystemisautonomoustoextentthatits
behaviorisdeterminedbyitsexperience
•  Asystemisn’tautonomousifguidedbyits
designeraccordingtoaprioridecisions
•  Anautonomousagentcanalwayssay“no”
•  Tosurvive,agentsmusthave:
– Enoughbuilt-inknowledgetosurvive
– Theabilitytolearn
Someagenttypes
(0)Table-drivenagents
simple
– Useperceptsequence/ac/ontabletofindthenextac/on.Implemented
bya(large)lookuptable
(1)Simplereflexagents
– Basedoncondi:on-ac:onrules,implementedwithanappropriate
produc/onsystem;statelessdeviceswithnomemoryofpastworldstates
(2)Agentswithmemory
– haveinternalstateusedtokeeptrackofpaststatesoftheworld
(3)Agentswithgoals
– Agentswithastateandgoalinforma:ondescribingdesirablesitua/ons.
Agentsofthiskindtakefutureeventsintoconsidera/on.
(4)U:lity-basedagents
– basedecisionsonclassicaxioma:cu:litytheoryinordertoactra/onally
complex
(0/1)Table-driven/reflexagentarchitecture
Useperceptsequence/ac/ontabletofindthenextac/on.
Implementedbya(large)lookuptable
(0)Table-drivenagents
Tablelookupofpercept-ac/onpairsmappingfrom
everypossibleperceivedstatetoop/malac/onforit
Problems:
– Toobigtogenerateandtostore(Chesshasabout
10120states,forexample)
– Noknowledgeofnon-perceptualpartsofthe
currentstate
– Notadap/vetochangesintheenvironment;en/re
tablemustbeupdatedifchangesoccur
– Looping:Can’tmakeac/onscondi/onalon
previousac/ons/states
(1)Simplereflexagents
•  Rule-basedreasoningmapsperceptstoop/mal
ac/on;eachrulehandlescollec/onofperceived
states(akareac/veagents)
•  Problems
– S/llusuallytoobigtogenerateandtostore
– S/llnoknowledgeofnon-perceptualpartsofstate
– S/llnotadap/vetochangesinenvironment;
collec/onofrulesmustbeupdatedifchangesoccur
– S/llcan’tcondi/onac/onsonpreviousstate
– Difficulttoengineerifthenumberofrulesislarge
duetoconflicts
(2)Architectureforanagentwithmemory
internalstateusedtokeeptrackofpaststatesof
theworld
(2)Agentswithmemory
•  Encodeinternalstateofworldtoremember
pastascontainedinearlierpercepts
–  Note:sensorsdon’tusuallygiveen/reworld
stateateachinput,soenvironmentpercep/on
iscapturedover0me.
–  Stateusedtoencodedifferent"worldstates"
thatgeneratethesameimmediatepercept
• Requiresrepresen0ngchangeintheworld
–  Wemightrepresentjustlateststate,butthen
can’treasonabouthypothe/calcoursesof
ac/on
(3)Architectureforgoal-basedagent
stateandgoalinforma:ondescribedesirablesitua/ons
allowingagenttotakefutureeventsintoconsidera/on
(3)Goal-basedagents
• Delibera:veinsteadofreac:ve
• Chooseac/onstoachieveagoal
• Goalisadescrip/onofadesirablesitua/on
• KeepingtrackofcurrentstateoEennot
enough:mustaddgoalstodecidewhich
situa/onsaregood
• Achievinggoalmayrequirelongac/on
sequence
–  Modelac/onconsequences:“whathappensifI
do...?”
–  Useplanningalgorithmstoproduceac/on
sequences
(4)acompleteu:lity-basedagent
basedecisionsonu/litytheoryinordertoactra/onally
(4)U:lity-basedagents
• Formul/plepossiblealterna/ves,howto
decidewhichisbest?
• Goalsgiveacrudedis/nc/onbetweenhappy
andunhappystates,butoEenneeda
performancemeasurefordegree
• U/lityfunc/onU:State→Realsgivesmeasure
ofsuccess/happinessforgivenstate
• Allowsdecisionscomparingchoicesbetween
conflic/nggoalsandlikelihoodofsuccessand
importanceofgoal(ifachievementuncertain)
Proper:esofEnvironments
• Fully/Par:allyobservable
– Ifagent’ssensorsgivecompletestateofenvironmentneeded
tochooseac/on,environmentisfullyobservable
– Suchenvironmentsareconvenient,freeingagentsfrom
keepingtrackoftheenvironment’schanges
• Determinis:c/Stochas:c
– Environmentisdeterminis:cifnextstateiscompletely
determinedbycurrentstateandagent’sac/on
– Stochas:c(i.e.,non-determinis/c)environmentshave
mul/ple,unpredictableoutcomes
• Infullyobservable,determinis/cenvironments
agentsneednotdealwithuncertainty
Proper:esofEnvironments
•  Episodic/Sequen:al
– Inepisodicenvironmentssubsequentepisodesdon’t
dependonac/onsinpreviousepisodes
– Insequen:alenvironmentsagentengagesinaseriesof
connectedepisodes
– Episodicenvironmentsdon’trequireagenttoplanahead
•  Sta:c/Dynamic
– Sta:cenvironmentsdoesn’tchangeasagentisthinking
– Thepassageof/measagentdeliberatesisirrelevant
– Theagentneedn’tobserveworldduringdelibera/on
Proper:esofEnvironmentsIII
• Discrete/Con:nuous
– Ifnumberofdis/nctperceptsandac/onsislimited,
environmentisdiscrete,otherwiseit’scon:nuous
• Singleagent/Mul:agent
– Inenvironmentswithotheragents,agentmust
considerstrategic,game-theore/caspectsof
environment(foreithercoopera/veorcompe//ve
agents)
– Mostengineeringenvironmentsdon’thave
mul/agentproper/es,whereasmostsocialand
economicsystemsgettheircomplexityfrom
interac/onsof(moreorless)ra/onalagents
Characteris:csofenvironments
Fully
Deterministic?
observable?
Episodic?
Static?
Discrete?
Single
agent?
Solitaire
Backgammon
Taxi driving
Internet
shopping
Medical
diagnosis
A Yes in a cell means that aspect is simpler; a No more complex
Characteris:csofenvironments
Solitaire
Fully
observable
Deterministic
Episodic
Static
Discrete?
Single
agent?
No
Yes
Yes
Yes
Yes
Yes
Backgammon
Taxi driving
Internet
shopping
Medical
diagnosis
A Yes in a cell means that aspect is simpler; a No more complex
Characteris:csofenvironments
Fully
Deterministic?
observable?
Episodic?
Static? Discrete?
Single
agent?
Solitaire
No
Yes
Yes
Yes
Yes
Yes
Backgammon
Yes
No
No
Yes
Yes
No
Taxi driving
Internet
shopping
Medical
diagnosis
A Yes in a cell means that aspect is simpler; a No more complex
Characteris:csofenvironments
Fully
Deterministic?
observable?
Episodic?
Static? Discrete?
Single
agent?
Solitaire
No
Yes
Yes
Yes
Yes
Yes
Backgammon
Yes
No
No
Yes
Yes
No
Taxi driving
No
No
No
No
No
No
Internet
shopping
Medical
diagnosis
A Yes in a cell means that aspect is simpler; a No more complex
Characteris:csofenvironments
Fully
Deterministic?
observable?
Episodic?
Static? Discrete?
Single
agent?
Solitaire
No
Yes
Yes
Yes
Yes
Yes
Backgammon
Yes
No
No
Yes
Yes
No
Taxi driving
No
No
No
No
No
No
Internet
shopping
No
No
No
No
Yes
No
Medical
diagnosis
A Yes in a cell means that aspect is simpler; a No more complex
Characteris:csofenvironments
→ Lots of real-world domains fall into the hardest case!
Fully
Deterministic?
observable?
Episodic?
Static?
Discrete?
Single
agent?
Solitaire
No
Yes
Yes
Yes
Yes
Yes
Backgammon
Yes
No
No
Yes
Yes
No
Taxi driving
No
No
No
No
No
No
Internet
shopping
No
No
No
No
Yes
No
Medical
diagnosis
No
No
No
No
No
Yes
A Yes in a cell means that aspect is simpler; a No more complex
Summary
• Anagentperceivesandactsinan
environment,hasanarchitectureandis
implementedbyanagentprogram
• Anidealagentalwayschoosesac/onthat
maximizesitsexpectedperformance,given
itsperceptsequencesofar
• Anautonomousagentusesitsown
experienceratherthanbuilt-inknowledgeof
environmentbydesigner
Summary
• Anagentprogrammapsperceptstoac/ons
andupdatesitsinternalstate
– Reflexagentsrespondimmediatelytopercepts
– Goal-basedagentsacttoachievetheirgoal(s)
– U:lity-basedagentsmaximizetheiru/lityfunc/on
• Represen:ngknowledgeisimportantfor
goodagentdesign
• Mostchallengingenvironmentsarepar/ally
observable,stochas/c,sequen/al,dynamic,
andcon/nuousandcontainmul/pleagents
Summary
• NotallAIproblemsagoodfitfororrequirean
agentagentmodel,e.g.,playingsolitaire
• NoraremanyAItasksyoumightbeaskedto
solve:
–  Classifymoviereviewsasnega/ve,neutralor
posi/ve
–  Locatefacesofpeopleinanimage
–  Anefficienttheoremprover
–  Learnpreferredthermostatseengsforeach
hourofeachdayofaweek