Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Distributed Decision Making
using Team Theory
Joel George
Department of Aerospace Engineering
Indian Institute of Science, Bangalore
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
The Definition
Team
A group of individuals with different information obtained
through their respective observations of the environment, who
take decision about something different but having common
goal and preferences and receive a common reward as a joint
result of all the individual decisions.
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
The Pioneers
Marschak and Radner
Jacob Marschak
J. Marschak, “Elements for a Theory of
Teams,” Management Science, vol. 1,
no. 2, Jan. 1955.
Roy Radner
R. Radner, “Team Decision Problems,”
The Annals of Mathematical Statistics,
vol. 33, no. 3, Sep. 1962.
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
The Team
Origins
Theory of Teams
special case of theory of games
basic to theory of optimal organization
Organization: for a given state of world, the individuals
1
2
3
make different observations and get different information
take different decisions based on their observations
have different preferences in terms of tastes, believes,
etc.
‘Team is an organization with members having same
preferences’ – Marschak
idea of ‘team’ and ‘decision makers’
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
The Team
Basic Ingredients
Individual members of a team differ w.r.t their
information
possibilities of actions
but agree in
goal and/or preferences
There is uncertainty about
state of the world
other members’ actions
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
A Mathematical Framework
Notations
s state of world; a description of environment so complete
that if true and known, the consequences of every action
would be known
yi information; signal received through the process of
observation
yi = ηi (s)
di decision; action taken based on the information
di = δi (yi ) = δi [ηi (s)]
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
A Mathematical Framework
For a team T = 1, 2, . . . , N
Information structure



η=

η1
η2
..
.
Team decision function








δ=

ηN
Jo — Team TheoryTeam Theory
δ1
δ2
..
.
δN
Team decision








d=

d1
d2
..
.
dN





Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
A Mathematical Framework
Payoff is a function of state and team decision
ω(s, d)
Objective: Maximize the expectation of payoff
(s is a random variable)
X
E [ω(s, d)] =
φ(s)ω(s, δ[η(s)])
s∈S
Goal: Choose the best team decision function δ to maximize
expected payoff for given information structure
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
Looking for Optimal Solutions
If δ̂ is an optimal solution then it is necessarily
person-by-person-optimal (pbpo)
If the expected payoff cannot be improved by changing only
one member’s decision function then such a decision is
person-by-person-optimal or coordinate-wise optimal.
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
Looking for Optimal Solutions
If δ̂ is an optimal solution then it is necessarily
person-by-person-optimal (pbpo)
If the expected payoff cannot be improved by changing only
one member’s decision function then such a decision is
person-by-person-optimal or coordinate-wise optimal.
Does pbp optimality imply a global optimum?
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
Looking for Optimal Solutions
Example
ω(d1 , d2 ) = min{−d12 − (d2 − 1)2 , (d1 − 1)2 − d2 }
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
Looking for Optimal Solutions
Example
ω(d1 , d2 ) = min{−d12 − (d2 − 1)2 , (d1 − 1)2 − d2 }
When does pbpo becomes sufficient for global optimality?
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
Looking for Optimal Solutions
Theorem 1 (Radner,1962)
If the team payoff function ω(s, d) is concave and
differentiable in decision variables di for every state s, then
person-by-person-optimal solution global optimum
Proof:
Let δ̂ be the pbpo solution
Define
k = (k1 , . . . , kN )
βi (s) = δi (s) − δ̂i (s), i = 1, . . . , N
f (s, k) = ω[s, δ̂1 (s) + k1 β1 (s), . . . , δ̂N (s) + kN βN (s)]
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Team Theory
Looking for Optimal Solutions
The expectation of cost function is then
X
F (k) =
φ(s)f (s, k)
s∈S
For a k of the form k = (0, 0, . . . , ki , . . . , 0)
F (k) ≤ F (0)
This along with differentiability implies
∂F (k)
= 0, ∀i = 1, . . . , N
∂ki k=0
Now since function F is also concave, k = 0 is globally
optimal.
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Teams
Quadratic Payoff Function
Teams with quadratic payoff function
ω(s, d) = λ(s) + 2µT (s)d − dT ν(s)d
Theorem 2 (Radner, 1962)
If ν is positive definite and symmetric, then for a given
information yi the optimal decision functions are uniquely
determined by
X
E (δj νij |yi ) = E (νi |yi ), i = 1, . . . , N
δi (yi )E (νii |yi ) +
j6=i
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Teams
Quadratic Payoff Function
Teams with quadratic payoff function
ω(s, d) = λ(s) + 2µT (s)d − dT ν(s)d
Theorem 2 (Radner, 1962)
If ν is positive definite and symmetric, then for a given
information yi the optimal decision functions are uniquely
determined by
X
δi (yi )E (νii |yi ) +
E (δj νij |yi ) = E (νi |yi ), i = 1, . . . , N
j6=i
Is it possible to get the form of decision function?
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Teams
Quadratic Payoff Function
ω(s, d) = λ(s) + 2µT (s)d − dT ν(s)d
Theorem 3 (Radner, 1962)
If ν is constant and involved random variables are normally
distributed with E (yi ) = 0 and Var (yi ) = 1, then the optimal
decision is given by
δi (yi ) = bi yi + ci
where bi and ci are solutions of linear system
X
X
νij Cov (yi , yj )bj = Cov (yi , µi ) and
νij cj = E (µi )
j
Jo — Team TheoryTeam Theory
j
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Teams
Quadratic Payoff Function
ω(s, d) = λ(s) + 2µT (s)d − dT ν(s)d
Theorem 3 (Radner, 1962)
If ν is constant and involved random variables are normally
distributed with E (yi ) = 0 and Var (yi ) = 1, then the optimal
decision is given by
δi (yi ) = bi yi + ci
where bi and ci are solutions of linear system
X
X
νij Cov (yi , yj )bj = Cov (yi , µi ) and
νij cj = E (µi )
j
j
Optimal decision is linear in information variables
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Teams
Polyhedral Payoff Function
Payoff function is concave and polyhedral
Team decision problem equivalent to linear programming
problem in the ‘decision function space’
Payoff function linear in decision variable (Linear Team)
Decision problem already in linear programming form
Linear payoff function
ω(s, d) =
X
fi (s)di
i
Objective is to maximize
E (ω(s, d)) =
subject to
X
i
Jo — Team TheoryTeam Theory
XX
s
fi (s)di φ(s)
i
gi (s)di ≤ c
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Optimal Teams
Polyhedral Payoff Function
Payoff function is concave and polyhedral
Team decision problem equivalent to linear programming
problem in the ‘decision function space’
Payoff function linear in decision variable (Linear Team)
X
Decision problem already in linear programming form
Linear payoff function
ω(s, d) =
fi (s)di
i
Objective is to maximize
E (ω(s, d)) =
subject to
X
XX
s
fi (s)di φ(s)
i
gi (s)di ≤ c
i
Linear Programming under uncertainty
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
using Team Theory
The Problem
P. B. Sujit, A. Sinha, and D. Ghose, “Team, Game, and Negotiation based Intelligent
Autonomous UAV Task Allocation for Wide Area Applications,” Innovations in
Intelligent Machines - 1, Studies in Computational Intelligence, vol. 70, Springer, 2007.
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
using Team Theory
A distributed decision problem with team theoretic flavor
Individual members of a team differ
w.r.t their
information
possibilities of actions
but agree in
goal and/or preferences
There is uncertainty about
state of the world
other members’ actions
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
Problem Details
N UAVs and M targets
state
s = {UAV position, target position, target value}
|
{z
}
random variable
target value, r ∈ {0, 0.5, 1}
decision of i th UAV
di = {di1 , di2 , . . . , di(mi +1) }
1
if UAV i attacks target j
dij =
0
otherwise
mi is number of targets in sensor range
di(mi +1) is search task for UAV i
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
Problem Formulation
ni number of UAVs in sensor range
Payoff function
ω=
ni m
i +1
X
X
Cij dij
i=1 j=1
Cij = Cij (s) is the cost of attacking j th target by i th UAV
Constraints
1
UAV can execute only one task at a time
m
i +1
X
dij = 1,
∀i = 1, . . . , ni
j=1
2
a target is not attacked by more than one uav
ni
X
Jo — Team TheoryTeam Theory
i=1
dij ≤ 1,
∀j = 1, . . . , mi
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
Modeling
Modeling the cost function
time left in mission
Cost of search =
total time of mission
promote search in the beginning of mission
Cost of attack = target value × w −
time to reach target
total time of mission
attack closer and higher value targets
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
Modeling
Accounting for other members’ decisions
ω=
ni m
i +1
X
X
Cij dij
i=1 j=1
Virtual UAVs
Virtual targets
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Task Allocation
Team Theoretic Formulation
Objective: Solve the Linear Programming problem
!
X
max E
Cij dij
d
i,j
subject to
X
j
Jo — Team TheoryTeam Theory
dij = 1,
X
i
dij ≤ 1
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Dynamic Teams
A Quick Look
Observation depends on other decision makers’ decisions
Optimal strategy exist only under assumptions of
1
2
Linear Quadratic Gaussian problem
Partially Nested information structure
Ex: Source Coding problem
Yu-Chi Ho, and K’ai-Ching Chu, “Team Decision Theory and Information Structures
in Optimal Control Problems – Part I,” IEEE Transactions on Automatic Control, vol.
17, no. 1, Feb 1972.
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
Summing Up
Definition of team
Mathematical framework for team problems
state
information structure
decision function
payoff function
Optimal team decisions
concavity and differentiability
quadratic and polyhedral payoff
Task allocation example
Dynamic teams
Jo — Team TheoryTeam Theory
Background Problem Formulation Optimal Solutions Illustration Extension Conclusion
In a Nutshell
Take home message
Team theory provides a suitable framework for the correct
statement of an informally decentralized optimization problem
and enables one to determine analytically the optimal
strategies in a few cases
Jo — Team TheoryTeam Theory