Intelligent Agents
– state of the art –
Aleksander Pivk
Materials collected at:
America School on Agents and Multi-agent Systems
(University of Southern California)
69th Solomon Seminar
What is an agent?
• The main point about agents is that they are
autonomous: capable of acting
independently, exhibiting control over their
internal state.
• Thus: an agent is a computer system
capable of autonomous action in some
environment.
SYSTEM
output
input
ENVIRONMENT
2/58
69th Solomon Seminar
What is an agent?
• Trivial (non-interesting) agents:
– thermostat
• An intelligent agent is a computer system capable
of flexible autonomous action in some
environment.
By flexible, we mean:
– reactive;
– pro-active;
– social.
3/58
69th Solomon Seminar
Intelligent Agents and AI
• – A little intelligence goes a long way! –
• Oren Etzioni, speaking of commercial experience of
NETBOT, Inc.:
We made our agents dumber and dumber
…until finally they made some money.
• Microsoft Office Assistant
4/58
69th Solomon Seminar
Purely Reactive Agents
• Some agents decide what to do without reference
to their history – they base their making decision
entirely on the present, with no reference at all to
the past.
• We call such agents purely reactive:
action: S A
• A thermostat is a purely reactive agent:
action(s)= off;
on;
if s=temperature OK
otherwise
5/58
69th Solomon Seminar
Perception
• Introduce the perception system:
see
input
action
AGENT
output
ENVIRONMENT
• The see function is the agent’s ability to observe
its env., whereas the action function represents the
agent’s decision making process.
• New functions:
see: S P
action: P* A
{maps environment states to percepts}
{maps sequences of percepts to actions}
6/58
69th Solomon Seminar
Agents with State
• Lets consider agents that maintain state:
AGENT
see
action
next
state
output
input
ENVIRONMENT
• Have some internal data structure, used to record
inf. about the env. state and history.
• Let I be the set of all internal states of the agent.
• Functions:
see: S P
{maps environment states to percepts}
action: I A
{maps from internal states to actions}
next: I  P  I {maps an internal states and percept to IS}
7/58
69th Solomon Seminar
Application & Research Domains
•
•
•
•
•
Learning Agents
Embodied Agents
Logics for Agents
Coordination, Cooperation, Collaboration
Market-based Multi-agent Systems
8/58
69th Solomon Seminar
Learning Agents
• Why should agents learn?
– Learning user and world models, action-to-utility
mappings, problem solving
• Learning modalities
– from users (observation, interaction, being told)
– from other agents (collaborative filtering, from experts)
– from experience (supervised, reinforcement,
probabilistic models)
• Learning techniques
– neural/decision networks, decision trees,reinforcement
learning, instance based learning
• Assistant agents (work effort, productivity)
9/58
69th Solomon Seminar
Learning for Information Agents
• Information agents
– Access information from a variety of data sources
– Integrate the data from these sources
– Monitor and provide notifications
• Technical challenges
–
–
–
–
Turning semi-structured data into structured data
Ensuring continued access to the data
Resolving naming inconsistencies across sources
Building agents that efficiently execute their tasks
10/58
69th Solomon Seminar
Country Information Agent
World
Governments
Agent
NATO Members
CIA World Factbook
1995
1996
1997
11/58
69th Solomon Seminar
Flight Delay Prediction Agent
Yahoo Weather
Prediction
Historical Flight
Data
Agent
Historical Weather
Data
Learned Flight
Delay Predictor
12/58
69th Solomon Seminar
Real Estate Notification Agent
Send Email
Notification
New Listing:
3br 2bath
200K
13/58
Travel Planning Agent
69th Solomon Seminar
14/58
69th Solomon Seminar
Wrapper Induction
Problem description:
• Web sources present data in human-readable format
– take user query
– apply it to data base
– present results in “template” HTML page
• To integrate data from multiple sources, one must first
extract relevant information from Web pages
• Task: learn extraction rules based on labeled examples
– Hand-writing rules is tedious, error prone, and time consuming
15/58
69th Solomon Seminar
Example of Extraction Task
NAME
STREET
CITY
PHONE
Casablanca Restaurant
220 Lincoln Boulevard
Venice
(310) 392-5751
16/58
69th Solomon Seminar
WIEN [Kushmerick et al ‘97, ‘00]
• Assumes items are always in fixed, known order
– … Name: J. Doe; 1 Main; 111-1111. <p> Name: E. Poe; …
• Introduces several types of wrappers
– LR:
Name:
Name
;
:
Addr
;
:
Phone
.
• Advantages:
– Fast to learn & extract
• Drawbacks:
– Cannot handle permutations and missing items
– Must label entire page
17/58
69th Solomon Seminar
SoftMealy [Hsu & Dung, ‘98]
• Learns a transducer
Name:
• Advantages:
Addr:
Addr
;
;
Name
;
– Also learns order of items
– Allows item permutations & missing items
– Uses wildcards (eg, Number, AllCaps, etc)
Phone:
Phone:
. Phone
• Drawback:
– Must “see” all possible permutations
18/58
69th Solomon Seminar
WHIRL Wrappers [Cohen ’99]
• Learns underlying HTML template
– WHIRL “soft logic” to measure document similarity
• Name:
html_table_tr_td
• Address: html_table_tr_td_td
• Advantages:
– Learns from unlabeled data
– Explicitly exploits HTML structure
• Disadvantage:
– Not as expressive as previous ones
– Works only at the level of “HTML nodes”
19/58
69th Solomon Seminar
STALKER [Muslea et al, ’98 ’99 ’01]
• Hierarchical wrapper induction
– Decomposes a hard problem in several easier ones
– Extracts items independently of each other
– Each rule is a finite automaton
• Advantages:
– Powerful extraction language (eg, embedded list)
– One hard-to-extract item does not affect others
• Disadvantage:
– Does not exploit item order (sometimes may help)
20/58
69th Solomon Seminar
Extraction Rules
Extraction rule: sequence of landmarks
SkipTo(Phone) SkipTo(<i>)
SkipTo(</i>)
Name: Joel’s <p> Phone: <i> (310) 777-1111 </i><p> Review: …
21/58
69th Solomon Seminar
More about Extraction Rules
Name: Joel’s <p> Phone: <i> (310) 777-1111 </i><p> Review: …
Name: Kim’s <p> Phone (toll free) : <b> (800) 757-1111 </b> …
Name: Kim’s <p> Phone:<b> (888) 111-1111 </b><p>Review: …
Start: EITHER SkipTo( Phone : <i> )
OR
SkipTo( Phone ) SkipTo(: <b>)
22/58
69th Solomon Seminar
Learning the Extraction Rules
GUI
Labeled Pages
Inductive
Learning
System
Extraction
Rules
EC Tree
23/58
69th Solomon Seminar
Example of Rule Induction
Training Examples:
Name: Del Taco <p> Phone (toll free) : <b> ( 800 ) 123-4567 </b><p>Cuisine ...
Name: Burger King <p> Phone : ( 310 ) 987-9876 <p> Cuisine: …
Initial candidate:
SkipTo( ( )
24/58
69th Solomon Seminar
Example of Rule Induction
Training Examples:
Name: Del Taco <p> Phone (toll free) : <b> ( 800 ) 123-4567 </b><p>Cuisine ...
Name: Burger King <p> Phone : ( 310 ) 987-9876 <p> Cuisine: …
Initial candidate:
SkipTo( <b> ( )
…
SkipTo( ( )
... SkipTo(Phone) SkipTo( ( ) ... SkipTo(:) SkipTo(()
SkipTo(Phone) SkipTo(:) SkipTo( ( )
...
25/58
69th Solomon Seminar
Multi-view Learning
Two ways to find start of the phone number:
SkipTo( Phone: )
BackTo( ( Number ) )
Name: KFC <p> Phone: (310) 111-1111 <p> Review: Fried chicken …
26/58
69th Solomon Seminar
Co-Testing
+
Labeled data
RULE 1
RULE 2
Unlabeled data
+
+
-
-
+
27/58
69th Solomon Seminar
Co-Testing for Wrapper Induction
SkipTo( Phone: )
BackTo( (Number) )
Name: Joel’s <p> Phone: (310) 777-1111 <p>Review: ...
Name: Kim’s <p> Phone: (213) 757-1111 <p>Review: ...
Name: Chez Jean <p> Phone: (310) 666-1111 <p> Review: ...
Name: Burger King <p> Phone: (818) 789-1211 <p> Review: ...
Name: Café del Rey <p> Phone: (310) 111-1111 <p> Review: ...
Name: KFC <p> Phone:<b> (800) 111-7171 </b> <p> Review:...
28/58
Embodied Agents
69th Solomon Seminar
Embodied Agents
• Animated agent research integrates:
– Artistic animation
– Computer graphics
– Intelligent agents
• Why build animated agents?
–
–
–
–
For more effective communication
For artistic effect
As models for robotic or human agents
When behavior cannot be scripted
• e.g., due to interactions with people and other agents
• In agents, we begin to see dynamic models of
thought and action
30/58
69th Solomon Seminar
Animated Pedagogical Agents
• Animated characters that:
–
–
–
–
–
Interact with students in learning envs.
Help keep learning on track
Act as guides, tutors, teammates
Engage in instructional dialog
Enhance motivation and interest
• APA’s require:
– Realistic, lifelike behavior
– A rich set of cognitive and social
abilities for effective learning
31/58
69th Solomon Seminar
Steve: An Embodied Intelligent
Agent for Virtual Environments
• J. Rickel, L. Johnson, M. Thiebaux, et al.
• 3D agent that interacts with students in virtual
environments
• Can work together with multiple students and
multiple users
32/58
69th Solomon Seminar
Steve’s Architecture (detailed)
Cognition
Steve
Domain knowledge
General capabilities
Motor commands
Motor
Control
Current state
Translate into
movements, speech
Filter, assemble
into coherent view
Broadcast to
environment
Monitor events
Commands to
environment
Perception
Event notifications
Virtual Environment
33/58
69th Solomon Seminar
Mission Rehearsal Exercise
34/58
69th Solomon Seminar
Co. & Co. & Co.
As a process
As an outcome
Coordinatio
n
Decisionmaking takes
others into account
Individuals are not
thwarted by others
Cooperation
Individual decisions
further the collective
welfare
Individuals appear to
be
“working together”
Collaboratio
n
Individuals work
toward shared goals
Common goals
achieved well and/or
efficiently
35/58
Example: Hidden Pictures
69th Solomon Seminar
• Simple (visual)
search task
• How would YOU
work as a part of a
team to solve it ?
36/58
69th Solomon Seminar
Market-based MAS
• Marketspace: class of agent interaction env.
• What you need to know:
– Economic foundations and principles:
• Game theory
• Price system (general equlibrium)
• Auction theory
– Design issues and experience
• Market models and mechanisms
• Trading Agents
37/58
69th Solomon Seminar
Business games
• http://www.cmu.edu/comlabgames
----------------------------------------------------------At the comlabgames website, www.cmu.edu/comlabgames, there are
three modules for designing, playing and analyzing the experimental
outcomes of games: two person strategic form games, multi-person
extensive form games, and auctions and markets. The original
comlabgames website is visited on average 50,000 of time each week,
and is linked to hundreds of other sites. UCLA mirrors the original site.
Comlabgames is very easy to use, and the students just bring their
laptops to class, design the games, run them and then analyze the data.
• Vesna Prasnikar, Marko Grobelnik
38/58
69th Solomon Seminar
Logics for Agents
• Symbolic Reasoning
– an agent contains an explicitly represented, symbolic
model of the world
– makes decisions (action to take) via symbolic reasoning
– problems:
• transduction: how to translate the real world in accurate,
adequate symbolic description (speech understanding)
• representation/reasoning: how to symbolically represent inf.,
and how to get agents to reason with this inf. (planning)
– Theorem Proving Agents: agent decides what to do by using
logic to encode the theory stating the best action in any situation
(predicates + rules)
– Agent oriented programming: AGENT0 and PLACA
39/58
69th Solomon Seminar
Logics for Agents
• Practical Reasoning (BDI Logic)
– is a matter of weighting conflicting considerations for and
against competing options, where the relevant
considerations are provided by the agent desires/values
about and what the agent believes (Bratman).
– directed towards actions (theoretical towards belief)
– consists of two activities:
• deliberation: deciding what state of affairs we want to achieve
• means-end reasoning: deciding how to achieve them
– implemented BDI agents: IRMA, PRS, Desiderata, LORA
40/58