INTELLIGENT AGENTS
1
DEFINITION OF AGENT

Anything that:
Perceives its environment
 Acts upon its environment


A.k.a. controller,
robot
2
DEFINITION OF “ENVIRONMENT”
The real world, or a virtual world
 Rules of math/formal logic
 Rules of a game
…
 Specific to the problem domain

3
Agent
Percepts
Sensors
Actions
Actuators
Environment
?
4
Agent
Percepts
Sensors
Actions
Actuators
Sense – Plan – Act
Environment
?
5
“GOOD” BEHAVIOR
Performance measure (aka reward, merit, cost,
loss, error)
 Part of the problem domain

6
EXERCISE

Formulate the problem domains for:
Tic-tac-toe
 A web server
 An insect
 A student in B551
 A doctor diagnosing
a patient
 IU’s basketball team
 The U.S.A.

What is/are the:
• Environment
• Percepts
• Actions
• Performance measure
How might a “goodbehaving” agent process
information?
7
TYPES OF AGENTS
Simple reflex (aka reactive, rule-based)
 Model-based
 Goal-based
 Utility-based (aka decision-theoretic, gametheoretic)
 Learning (aka adaptive)

8
SIMPLE REFLEX
Percept
Interpreter
State
Rules
9
Action
SIMPLE REFLEX
Percept
Rules
10
Action
SIMPLE REFLEX
Percept
In observable
environment,
percept = state
Rules
11
Action
RULE-BASED REFLEX AGENT
A
B
if DIRTY = TRUE then SUCK
else if LOCATION = A then RIGHT
else if LOCATION = B then LEFT
12
BUILDING A SIMPLE REFLEX AGENT

Rules: a map from states to action


a = (s)
Can be:
Designed by hand
 Learned from a “teacher” (e.g., human expert) using
ML techniques

13
MODEL-BASED REFLEX
Percept
Interpreter
State
Rules
Action
14
Action
MODEL-BASED REFLEX
Percept
Model
State
Rules
Action
15
Action
MODEL-BASED REFLEX
Percept
Model
State
State
estimation
Rules
Action
16
Action
MODEL-BASED AGENT
State:
LOCATION
HOW-DIRTY(A)
Rules:
if LOCATION = A then
if HAS-SEEN(B) = FALSE then RIGHT
HOW-DIRTY(B)
else if HOW-DIRTY(A) > HOW-DIRTY(B) then SUCK
HAS-SEEN(A)
HAS-SEEN(B)
A
else RIGHT
…
B
Model:
HOW-DIRTY(LOCATION) = X
HAS-SEEN(LOCATION) = TRUE
17
MODEL-BASED REFLEX AGENTS
Controllers in cars, airplanes, factories
 Robot obstacle avoidance, visual servoing

18
BUILDING A MODEL-BASED REFLEX
AGENT

A model is a map from prior state s, action a, to
new state s’


s’ = T(s,a)
Can be
Constructed through domain knowledge (e.g., rules of
a game, state machine of a computer program, a
physics simulator for a robot)
 Learned from watching the system behave (system
identification, calibration)


Rules can be designed or learned as before
19
GOAL-BASED, UTILITY-BASED
Percept
Model
State
Rules
Action
20
Action
GOAL-BASED, UTILITY-BASED
Percept
Model
State
Decision Mechanism
Action
21
Action
GOAL-BASED, UTILITY-BASED
State
Decision Mechanism
Action Generator
Percept Model
Model
Simulated State
Performance tester
Best Action
22
Action
GOAL-BASED, UTILITY-BASED
State
Decision Mechanism
Action Generator
“Every good regulator of a system
must be a model of that system”
Sensor Model
Model
Simulated State
Performance tester
Best Action
23
Action
BUILDING A GOAL OR UTILITY-BASED
AGENT

Requires:
Model of percepts (sensor model)
 Action generation algorithm (planner)
 Embedded state update model into planner
 Performance metric


Model of percepts can be learned (sensor
calibration) or approximated by hand (e.g.,
simulation)
24
BUILDING A GOAL-BASED AGENT

Requires:
Model of percepts (sensor model)
 Action generation algorithm (planner)
 Embedded state update model into planner
 Performance metric

Model of percepts can be learned (sensor
calibration) or approximated by hand (e.g.,
simulation)
 Planning using search
 Performance metric: does it reach the goal?

25
BUILDING A UTILITY-BASED AGENT

Requires:
Model of percepts (sensor model)
 Action generation algorithm (planner)
 Embedded state update model into planner
 Performance metric

Model of percepts can be learned (sensor
calibration) or approximated by hand (e.g.,
simulation)
 Planning using decision theory
 Performance metric: acquire maximum rewards
(or minimum cost)

26
BIG OPEN QUESTIONS:
GOAL-BASED AGENT = REFLEX AGENT?
Physical Environment
“Mental Environment”
Percept
Model
Mental Model
State
Mental State
DM Rules
Action
Mental Action
27
Action
BIG OPEN QUESTIONS:
GOAL-BASED AGENT = REFLEX AGENT?
Physical Environment
“Mental Environment”
Percept
Model
Mental Model
State
Mental State
DM Rules
Action
Mental Action
28
Action
WITH LEARNING
Percept
Model/Learning
State/Model/DM specs
Decision Mechanism
Action
29
Action
BUILDING A LEARNING AGENT
Need a mechanism for updating
models/rules/planners on-line as it interacts
with the environment
 Need incremental techniques for machine
learning
 More next week…

30
BIG OPEN QUESTIONS: LEARNING AGENTS
The modeling, learning, and decision
mechanisms of artificial agents are tailored for
specific tasks
 Are there general mechanisms for learning?
 If not, what are the limitations of the human
brain?

31
TYPES OF ENVIRONMENTS
Observable / non-observable
 Deterministic / nondeterministic
 Episodic / non-episodic
 Single-agent / Multi-agent

32
OBSERVABLE ENVIRONMENTS
Percept
Model
State
Decision Mechanism
Action
33
Action
OBSERVABLE ENVIRONMENTS
State
Model
State
Decision Mechanism
Action
34
Action
OBSERVABLE ENVIRONMENTS
State
Decision Mechanism
Action
35
Action
NONDETERMINISTIC ENVIRONMENTS
Percept
Model
State
Decision Mechanism
Action
36
Action
NONDETERMINISTIC ENVIRONMENTS
Percept
Model
Set of States
Decision Mechanism
Action
37
Action
AGENTS IN THE BIGGER PICTURE
Binds disparate fields (Econ,
Cog Sci, OR, Control theory)
 Framework for technical
components of AI




Decision making with search
Machine learning
Casting problems in the
framework sometimes brings
insights
Agent
Robotics
Reasoning
Search
Perception
Learning
Knowledge Constraint
rep.
satisfaction
Planning
Natural
language
...
Expert
Systems
UPCOMING TOPICS
Utility and decision theory (R&N 17.1-4)
 Reinforcement learning


Project midterm report due next week (11/10)
39