Team Decision Theory and Information Structures in
Optimal Control Problems
YU-CHI HO and K’AI-CHING CHU
Presented by: Ali Zaidi, Comm. Theory, KTH
October 4, 2011
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
1 / 13
Problem Setup
Team: A set of decision makers which have a common goal,
I = {1, 2, · · · , 3}.
The member i ∈ I receives information zi and makes a decision
ui = γi (zi ), where γi ∈ Γi .
A common pay-off: J(γ1 , γ2 , · · · , γN ).
Problem 1: Find γi? ∈ Γi for all i such that J is minimized.
Static vs Dynamic Teams.
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
2 / 13
Information Structures
Let ξ ∈ Rn represent uncertainties of the external world, with pdf
ξ ∼ N (0, X ) known to all members.
The information variable zi = Hi ξ +
P
j
Dij uj .
Interested in causal systems: Dij 6= 0 ⇒ Dji = 0.
Definition 1: We say j is related to i, jRi, if Dij 6= 0.
Definition 2: We say j is precedent of i, j{i, iff (a) jRi or (b) there
exist distinct r , s, · · · , k ∈ I such that jRr and rRs, ..., kRi.
Precedence diagram to graphical represent the idea of precedence.
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
3 / 13
Information Structures-Example 1
Example 1:
Dij = 0,
zi = Hi ξ,
∀i and j
∀i
(1)
Static or dynamic team?
Precedence diagram.
No causal precedence relation among DMs, therefore actual time
instants of observation and decisions are not important.
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
4 / 13
Information Structures-Example 2
Example 2 (Classical multi-stage stochastic control) : A dynamic LTI
system (organization)
xi+1 = Fxi + Gui + wi ,
yi = Hxi + vi ,
i = 1, 2, · · · , N,
i = 1, 2, · · · , N,
(2)
(3)
where xi is the state, ui the control, and wi , vi independent
sequences.
Perfect memory system– Each DM remembers it’s own past
observations and decisions.
Express information
at the i − th DM in the following form:
P
zi = Hi ξ + j Dij uj .
Draw the precedence diagram with.
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
5 / 13
Pay-off Function and Control Laws
Problem P1: The common goal for all members is to minimize
J(γ1 , · · · , γN ) = E[J ] = E[uT Qu + uT Sξ + uT c],
where u = [u1 u2 · · · uN ]T , ui = γi (zi ), γi ∈ Γi , i ∈ {1, 2, · · · , N}.
Problem P2 (Member-by-member optimality): Find γi? ∈ Γi for all i
such that
?
?
?
?
J(γ1 , · · · , γi−1
, γi? , γi+1
, · · · , γN? ) ≤ J(γ1 , · · · , γi−1
, γi , γi+1
, · · · , γN? )
for all γi ∈ Γi and for all i.
Problem P3: For fixed γj? (zj ), j 6= i, and any zi
min E[uT Qu + uT Sξ + uT c] =: min Ji
ui
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
(4)
ui
October 4, 2011
6 / 13
Optimal control laws for STATIC TEAMS
A linear information structure: zi = Hi ξ for all i.
Theorem (due to Radner): The optimal control law for each DM is
unique and affine in it’s observation variable. That is of the form,
ui = γi (zi ) + bi .
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
7 / 13
Dynamic Teams with Partially Nested Information
Structure
An information structure is called partially nested if j{i implies
Zj ⊂ Zi for all i, j, and γ.
Theorem 1: In a dynamic team with partially nested information
structure,
X
z i = Hi ξ +
Dij uj
(5)
j{i
is equivalent to an information structure in static form for any fixed
set of control laws:
ẑi = [{Hi ξ|j{i or j = i}] .
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
(6)
October 4, 2011
8 / 13
Partially Nested Information Structure-Example
Example:
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
9 / 13
Optimal control laws for Partially Nested Information
Structure
Theorem 2: In a dynamic team with partially nested information
structure, the optimal control for each member exists, is unique and
linear. How to prove?
Application 1: LQG Control Problem.
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
10 / 13
Application 2: One-step Communication Delay Control
System
Two coupled linear discrete-time dynamic systems controlled by u1 (t)
and u2 (t), t = 1, 2, · · · , N
The two controllers observe:
z1 (t) = {y1 (τ ), y2 (k)|τ = 1, 2, · · · , t; k = 1, 2, · · · , t − 1}
z2 (t) = {y2 (τ ), y1 (k)|τ = 1, 2, · · · , t; k = 1, 2, · · · , t − 1}
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
11 / 13
Further applications: Interconnected/ Hierarchical System
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
12 / 13
Take-home message
In a decentralized decision-making environment, if a DM’s action effects
our information, then knowing what he knows will yield linear
optimal solutions.
Thank you for your attention!
YU-CHI HO and K’AI-CHING CHU Presented by: Ali Zaidi, Comm. Theory, KTH ()
October 4, 2011
13 / 13