Multiagent Teamwork:
Analyzing the Optimality and Complexity of
Key Theories and Models
David V. Pynadath and Milind Tambe
Information Sciences Institute and
Department of Computer Science
University of Southern California
1
Agent Teamwork
Agents, robots, sensors, spacecraft, etc.





Performing a common task
Operating in an uncertain environment
Distinct, uncertain observations
Distinct actions with uncertain effects
Limited, costly communication
Battlefield Simulation
Satellite Clusters
Disaster Rescue
2
Motivation
Performance
?
Optimal
New algorithm
Optimal
? Practical
Systems
? Theoretical
Approaches
? No communication
Outline of Results
1) Unified teamwork framework
2) Complexity of optimal teamwork
3) New coordination algorithm
4) Optimality-Complexity evaluation
of existing methods
Complexity
3
Example Domain:
Helicopter Team
Did they
see that?
I destroyed the
enemy radar.
Enemy Radar
Goal
4
Communicative Multiagent Team
Decision Problem (COM-MTDP)
S: states of the world
 e.g., position of helicopters, position of the enemy
A: domain-level actions
 e.g., fly below radar, fly normal altitude
P: transition probability function
 e.g., world dynamics, effects of actions
S: communication capabilities, possible “speech acts”
 e.g., “I have destroyed enemy radar.”
5
COM-MTDPs (cont’d)
W: observations
 e.g., enemy radar, position of other helicopter
O: probability (for each agent) of observation
 Maps state and actions into distribution over
observations (e.g., sensor noise model)
R: reward (over states, actions, messages)

e.g., good if we reach destination, better if we reach it earlier
 e.g.,
saying, “I have destroyed enemy,” has a cost
Teamwork Definition:
 All members share same preferences (i.e., R)
6
Problem Complexity
No communication
Individually Observable
Collectively Observable
Free communication
COM-MTDPs
7
To Communicate or
Not To Communicate
Local decision of one agent at a single point in time:
 “I have achieved a joint goal.”
 “Should I tell my teammate?”
Joint intentions theory: “I must attain mutual belief.”
 Always communicate [Jennings]
STEAM:
 “I must communicate if the expected cost of
miscoordination outweighs the cost of
communication.” [Tambe]
 Each cost is a fixed parameter specified by designer
8
Locally Optimal Criterion
for Communication
Communicate if and only if:
 E[R
| communicate]  E[R | do
Expectation over
possible histories of
and beliefs
not states
communicate]
up to current time
Expected reward over
future trajectories
of states and beliefs
WITH communication
Expected reward over
future trajectories
of states and beliefs
WITHOUT communication
Expected cost of
communicating
9
Empirical Results
V_opt-V
Communication Cost
Observability
10
Empirical Results
11
Empirical Results
12
Optimality vs. Complexity
Optimality
E[R]
1.46
Locally Optimal
Globally Optimal
STEAM
Silent
1.43
1.40
Jennings
0.1
1.0
Observability = 0.2
Comm. Cost = 0.7
Complexity
10,000
seconds
(log)
13
Optimality vs. Complexity
Optimality
E[R]
STEAM
1.80
Jennings
Locally Optimal
1.43
Observability = 0.2
Comm. Cost = 0.3
Silent
0.1
Globally Optimal
1.0
Complexity
10,000
seconds
(log)
14
Summary
COM-MTDPs provide a unified framework for agent teamwork
 Representation subsumes many existing agent models
 Policy space subsumes many existing prescriptive theories
This framework supports deeper analyses of teamwork problems
 Quantitative characterization of optimality-efficiency
tradeoff , for different policies, in different domains
 Derivation of novel coordination algorithms
http://www.isi.edu/teamcore/Teamwork
 Detailed proofs
 Source code
 JAIR article
15