Task Planning and
Multi-Agent Systems!
•!
•!
•!
•!
•!
Robert Stengel!
Robotics and Intelligent Systems,
MAE 345, Princeton University, 2015
Decision making
Task decomposition, communities, and connectivity
Cooperation, collaboration, competition, and conflict
Path planning (see Lecture 5)
Multi-agent architectures
Copyright 2015 by Robert Stengel. All rights reserved. For educational use only.
http://www.princeton.edu/~stengel/MAE345.html
1
Task Planning Goals
•!
Accomplish an objective
•!
Determine and follow a path
•!
Work toward a common goal
–!
–!
–!
–!
–!
Central Pacific and Union Pacific Railroads meet in
Promontory, Utah, 1869
Make a decision
Gather information
Build something
Analyze something
Destroy something
–! Minimize time or cost
–! Take the shortest path
–! Avoid obstacles or hazards
–! Integrate behavior with higher
objectives
–! Do not impede other agents
Driving The Golden Spike
2
More Task Planning Goals
•! Provide leadership for other
agents
–! Issue commands
–! Receive and decode
information
•! Provide assistance to other
agents
–! Coordinate actions
–! Respond to requests
•! Defeat opposing agents
–! Compete and win
•! Path planning
–! See Lecture 5
3
Common Threads in
Task Accomplishment
•!
•!
•!
•!
•!
•!
•!
•!
Optimize a cost function
Satisfy or avoid constraints
Exhibit desirable behavior
Tradeoff individual and team goals
Use resources effectively and efficiently
Negotiate
Cooperate with team members
Overcome adversity and ambiguity
4
Task Planning
•! Situation awareness
•! Decomposition and identification of communities
•! Development of strategy and tactics
Objective
Tactical
(short-term)
Strategic
(long-term)
Phase
Process
Situation
Assessment
Comprehension
Outcome
Situation
Awareness
Understanding
5
Boyd
s OODA Loop
for Combat Operations
•! Derived from air-combat
maneuvering strategy
•! General application to learning
processes other than military
6
Elements of
Situation Awareness
Endsley, 1995
Important Dichotomies
in Planning
Strength, Weakness, Opportunity, and
Threat (SWOT) Analysis
Knok-Knoks and Unk-Unks
8
7
Strategy/Tactics Development and
Deployment
•! Development of long- and short-term
actions/activities for implementation and
operation
•! Sequence of procedures to be executed
–! fixed or adaptive
•! Exposition of approach
–! Rules of engagement
–! Concept of Operations (CONOPS)
•! Spectrum of flexibility
–! Rigid sequence <---> Learning systems
•! Think Expert System
9
Planning Tools!
10
Program Management:
Gantt Chart
•!
•!
•!
•!
Project schedule
Task breakdown and dependency
Start, interim, and finish elements
Time elapsed, time to go
11
Program Evaluation and Review
Technique (PERT) Chart
•!
•!
•!
•!
•!
•!
Milestones
Path descriptors
Activities, precursors, and successors
Timing and coordination
Identification of critical path
Optimization and constraint
12
Task Decomposition:
Community Identification
•! Connectivity of
individuals
•! Individuals
assemble in
communities or
clusters
•! Complex
networks
Fully connected
Small world ring lattice
–! Random
networks
–! Small-world
networks
–! Scale-free
networks
Random
Clustered small world
•! Degrees of
separation
Strogatz, 2001
Community <-> Communication
13
Communities and
Networks!
14
Scale-Free Networks
Frequency and cumulative distributions of cluster
sizes, k, inversely proportional to kx, x ~ –2 or –3
No “knee” that implies a scale in the distribution
Scale-Free
Strogatz, 2001
https://en.wikipedia.org/wiki/Scale-free_network
15
Community Examples
•! Families
•! Classmates
•! Neighbors
•! Social Networks
–! Facebook
–! LinkedIn
•! Media Networks
•! Corporations
•! Employees
•! Customers
•! Sports Leagues
–! Teams
•! Managers
–! Players
–! Trainers
•! Airlines
•! Cities
•! Associations
•! Governments
–! Agencies
•! Laboratories
–! Managers
–! Scientists
•! Military organizations
–! Army
•! Corps
–! Division
»! Brigade
–! Regiment
•! Battalion
–! Company
»! Platoon
–! Squad
•! Soldier
–! Special Operations
•! Terrorist organizations
16
Multi-Agent Systems
•! Specialized vs. general-purpose agents
•! Organizational models
•! Cooperators
–! Leader/follower (hierarchical)
–! Equal members
•! Collaborators
–! Air, ground, and sea traffic
–! Customers
•! Competitors
–! Individual game players
–! Sports teams
–! Political/military organizations
•! Negotiators
–! Politicians
–! Employer/employee representatives
17
Multi-Agent Systems
•! Cooperation and collaboration should
lead to win-win
(non-zero-sum)
solutions
•! Competition should lead to winlose
(zero-sum) solutions
•! Negotiation should lead to win-win
but
may lead to win-lose
solutions
18
Typical Characteristics of MultiAgent Architectures
•! Federated (centralized)
problem solving
–!
–!
–!
–!
–!
–!
–!
–!
–!
–!
–!
Doctrinaire
Coupled
Synchronous
Fragile
Complex
Strategic
Information-rich
Unified
Integrated
Top-down
Globally optimal
•! Distributed problem
solving
–!
–!
–!
–!
–!
–!
–!
–!
–!
–!
–!
Autonomous
Independent
Asynchronous
Robust
Simple
Tactical
Parsimonious
Idiosyncratic
Modular
Bottom-up
Locally optimal
19
Hierarchical Tree or
Hub-and-Spoke Network?
20
What is the Nature, Quality, and
Significance of Connections?
•!Communication
•!Collaboration
•!Coordination
•!Negotiation
•!Competition
•!Conflict
21
Connections May Connote
Different Relationships
•!Communication
•!Collaboration
•!Coordination
•!Negotiation
•!Competition
•!Conflict
22
Competition
23
Conventional Conflict
24
Unconventional
(
Asymmetric) Conflict
25
System Analysis of
the 9/11 Terrorist
Network
•!Hijackers
–!AA11
–!AA77
–!UA93
–!UA175
•!Accomplices
http://pear.accc.uic.edu/ojs/index.php/fm/article/view/941/863
26
27
Multi-Agent Scenarios
Modeled as Optimal
Control Problems!
28
A Federated Optimization Problem
•!
Dynamic models for two agents, A and B, are coupled to each
other and expressed as a single system
! F
A
x! (t) = Fx(t) + Gu(t) = # B
# FA
"
•!
FBA $ ! x A
&#
FB & #" x B
%
$ ! GA
&+# B
&% #" G A
$
&
&%
Cost function minimizes performance-control tradeoff
&( 1 t f
*(
E(J ) = E ' % !" xT (t)Qx(t) + uT (t)Ru(t) #$dt +
() 2 to
(,
& tf !
! Q QA # ! x #
(1
A
B
. - A . + ! u AT
= E ' % - ! xTA xTB # - B
"
$
2
. -" x B .$ "
Q
Q
B
() to "
$
" A
•!
G AB $ ! u A
&#
G B & #" u B
%
! R
A
u BT # - B
$- R
" A
R AB # ! u A
.R B . -" u B
$
# # *(
. .dt +
.$ . (
$ ,
Optimal feedback control laws are coupled to each other
" uA
u(t) = !Cx̂(t) = $
$# u B
" C
%
A
' = !$ B
$ CA
'&
#
C AB % " x̂ A
'$
C B ' $# x̂ B
&
%
'
'&
29
A Distributed Optimization Problem
•!
Coupling between actions of two agents,
A and B, is negligible
! FA
x! (t) = Fx(t) + Gu(t) = #
#" 0
0 $ ! xA
&#
FB & # x B
%"
$ ! GA
&+#
&% #" 0
&( 1 t f T
*(
E(J ) = E ' % !" x (t)Qx(t) + uT (t)Ru(t) #$dt +
() 2 to
(,
& tf !
! QA 0 # ! x A #
(1
.. + ! u AT
= E ' % - ! xTA xTB # "
$
"
0
Q
x
2
-"
B .() to "
$ " B .$
•!
•!
0 $ ! uA
&#
G B & # uB
%"
! RA
u BT # $- 0
"
$
&
&%
0 # ! uA
.R B . - uB
$"
# # *(
. .dt +
.$ .$ (
,
Each sub-system can be optimized separately
Each control depends only on separate sub-state
" RA
u(t) = ! $
$# 0
!1
" uA
0 %
' GT Sx̂(t) = !Cx(t) = $
RB '
$# u B
&
%
" CA
' = !$
'&
$# 0
0 % " x̂ A
'$
C B ' $ x̂ B
&#
%
'
'&
30
Pursuit-Evasion:!
A Competitive
Optimization Problem
•! Pursuers goal: minimize final miss distance
•! Evaders goal: maximize final miss distance
•! Linear model with two competitors, P and E
! x! P
x! (t) = Fx(t) + Gu(t) = #
#" x! E
•!
$ ! FP
&=#
&% #" 0
0 $ ! xP
&#
FE & # x E
%"
$ ! GP
&+#
&% #" 0
0 $ ! uP
&#
G E & # uE
%"
$
&
&%
Example of a differential game, Isaacs (1965), Bryson & Ho (1969)
31
Pursuit-Evasion:!
A Competitive
Optimization Problem
•! Quadratic minimax (saddle-point) cost function
tf
1
(& 1
(*
E(J ) = E ' !" xT (t f )S(t f )x(t f ) #$ + % "! xT (t)Qx(t) + uT (t)Ru(t) #$dt +
2
2
to
)(
,(
&
! S
S PE #
( 1 ! xTP (t f ) xTE (t f ) # - P
.
.
$ - S EP S E .
(2 "
$f
"
(
= E'
! QP QPE
( 1 tf !
(+ % - ! xTP (t) xTE (t) # $ - QEP QE
() 2 to -" "
"
! x P (t f )
- x E (t f )
"
#
.
.
$
# ! x P (t) #
! RP
. + ! uPT (t) uET (t) # ."
$
.$ -" x E (t) .$
-" 0
0
/R E
*
(
(
(
+
# ! uP (t) # # (
. .dt (
..$ -" uE (t) .$ .$ (
,
•! Optimal control laws for pursuer and evader
! u P (t) $
! C P (t) C PE (t) $ ! x̂ P (t) $
& = '#
&#
&
u(t) = #
#" u E (t) &%
#" C EP (t) C E (t) &% #" x̂ E (t) &%
32
Coordination
33
Collaboration
34
Decomposition into Fast and
Slow Models!
35
Reduction of Dynamic Model
Order
•! Separation of high-order models into loosely coupled or
decoupled lower order approximations
" !!x fast
$
$# !!x slow
% " Ffast
'=$
slow
'& $ Ffast
#
" Ff
=$
$# small
fast %
" !x fast
Fslow
'$
Fslow ' $# !x slow
&
small % " !x f
'$
Fs ' $ !x s
&#
% " G fast
'+$
'& $ G fast slow
#
% " Gf
'+$
'& $# small
G slow fast % " !u fast
'$
G slow ' $# !u slow
&
small % " !u f
'$
G s ' $ !u s
&#
%
'
'&
%
'
'&
36
Truncation of a Dynamic Model
•!
Dynamic model order reduction when
–! Two modes are only slightly coupled
–! Time scales of motions are far apart
–! Forcing terms are largely independent
" !!x f
$
$# !!x s
% " Ff
'=$
'& $ Ffs
#
Fsf % " !x f
'$
Fs ' $# !x s
&
" Ff
=$
$# small
" Ff
($
$# 0
•!
G sf % " !u f
'$
G s ' $# !u s
&
% " Gf
'+$
'& $ G sf
#
small % " !x f
'$
Fs ' $ !x s
&#
% " Gf
'+$
'& $# small
0 % " !x f % " G f
'$
'+$
Fs ' $ !x s ' $ 0
&#
& #
%
'
'&
small % " !u f
'$
G s ' $ !u s
&#
%
'
'&
0 % " !u f %
'$
'
G s ' $ !u s '
&#
&
Approximation: Modes can be analyzed and
control systems can be designed separately
!!x f = Ff !x f + G f !u f
!!x s = Fs !x s + G s !u s
37
Residualization of a Dynamic Model
•!
Dynamic model order reduction when
–! Two modes are coupled
–! Time scales of motions are separated
–! Fast mode is stable
" !!x f
$
$# !!x s
% " Ff
'=$
'& $ Ffs
#
" Ff
=$
$# small
•!
Fsf % " !x f
'$
Fs ' $# !x s
&
small % " !x f
'$
Fs ' $ !x s
&#
% " Gf
'+$
'& $ G f s
#
% " Gf
'+$
'& $# small
G s f % " !u f
'$
G s ' $# !u s
&
%
'
'&
small % " !u f
'$
G s ' $ !u s
&#
%
'
'&
Approximation: Motions can be analyzed separately
using different clocks
–! Fast mode reaches steady state instantaneously
on slow-mode time scale
–! Slow mode produces slowly changing bias
disturbances on fast-mode time scale
38
Residualized Fast Mode
Fast mode dynamics
!!x f = Ff !x f + G f !u f
+ ( Fsf !x s + G f s !u s )"Bias
If fast mode is not stable, it could be stabilized by
inner loop
control
(
!!x f = Ff !x f + G f !u c " C f !x f
(
+ F !x s + G !u s
(
f
s
)
f
s
)
)
" Bias
= Ff " G f C f !x f + G f !u fc
(
+ F !x s + G sf !u s
Fast Mode
Inner Loop
Control Law
f
s
)
" Bias
39
Fast Mode in Quasi-Steady State
Assume that fast mode reaches steady state on a time
scale that is short compared to the slow mode
0 ! Ff "x f + Fsf "x s + G f "u f + G sf "u s
"!x s = Ffs "x f + Fs "x s + G s "u s + G sf "u f
Algebraic solution for fast variable
0 ! Ff "x f + Fsf "x s + G f "u f + G sf "u s
Ff "x f = #Fsf "x s # G f "u f # G sf "u s
(
"x f = #Ff#1 Fsf "x s + G f "u f + G sf "u s
)
40
Residualized Slow Mode
Substitute quasi-steady fast variable in
differential equation for slow variable
(
)
!!x s = "Ffs #$ Ff "1 Fsf !x s + G f !u f + G f s !u s %& + Fs !x s + G s !u s + G s f !u f
= #$ Fs " Ffs Ff "1Fsf %& !x s + #$G s " Ffs Ff "1G sf %& !u s + #$G sf " Ffs Ff "1G f %& !u f
Residualized equation for slow variable
!!x s = FsNEW !x s + G sNEW
" !u f
$
$# !u s
%
'
'&
Slow Mode
Outer Loop
Control Law
Control law can be designed for reduced-order slow model,
assuming inner loop has been stabilized separately
41
Air Traffic Management: A
Collaborative Multi-Agent System
https://www.flightradar24.com
42
Elements of
Principled Negotiation
[Fisher, Ury (1981) Fry (1991)]
•!
•!
•!
•!
•!
Example of decision-making
Separate agents* from the problem
Focus on interests, not positions
Invent options for mutual gain
Insist on using objective criteria
* people, organizations, entities, …
43
Intelligent Agents in Air
Traffic Management
44
Principled Negotiation Flow Chart
45
Expert System
Diagram for
Principled
Negotiation
(Wangermann and Stengel)
•!
•!
•!
•!
Separate agents* from the
problem
Focus on interests, not
positions
Invent options for mutual
gain
Insist on using objective
criteria
46
Graphical
Representation of
Knowledge:
Principled
Negotiation in Air
Traffic
Management
47
Principled Negotiation:
Getting Past No (Ury, 1991)
•! Prepare by identifying barriers to cooperation, options, standards,
and your Best Alternative to a Negotiated Agreement (BATNA)
•! Understand your goals, limits, and acceptable outcomes
•! Buy time to think
, deflect attacks
•! Know your hot buttons
•! Acknowledge opposing arguments
•! Agree when you can without conceding
•! Express your views without provoking
•! I
statements, not you
statements
•! Negotiate the rules of the game
•! Reframe the negotiation
•! Build a golden bridge
that allows opponent to retreat gracefully
•! Engage third-party mediation or arbitration
•! Aim for mutual satisfaction, not victory
•! Forge a lasting agreement
48
Supplementary Material
49
Intelligent Aircraft/Airspace System
Flow Control
50
Intelligent Aircraft/Airspace System
Departure Control
51
A Cooperative MultiAgent System
52
Multi-Agent Control Example Based on
Linear-Quadratic-Gaussian !
(LQG) Optimal Control
•! Linear dynamic model
x! (t) = Fx(t) + Gu(t) + Lw(t)
•! Quadratic cost function
tf
+
'
)
)! "# x(t f ) $% + & L [ x(t), u(t)]dt
)
)
to
E(J ) = E (
,
tf
+
1 ') T
)
)
)
= (x (t f )S f x(t f ) + & "# xT (t)Qx(t) + uT (t)Ru(t) $%dt , )
)
2 *)
to
-) )*)
53
Conclusion
•!
Robots and Robotics
•!
Intelligent Systems
•!
Robots + Intelligent Systems = Intelligent Robotics
–!
–!
–!
–!
Mechanical devices
Design of mechanical devices
Use of mechanical devices
Control processes, sensors, and algorithms used in humans,
animals, and machines
–! Systems to perform useful functions driven by goals and
current knowledge
–! Systems that emulate biological and cognitive processes
–! Systems that process information to achieve objectives
–! Systems that learn by example
–! Systems that adapt to a changing environment
–! Optimization
54
MAE 345 Course Learning Objectives!
!!
!!
!!
!!
!!
!!
!!
!!
!!
!!
Dynamics and control of robotic devices.!
Cognitive and biological paradigms for system design. !!
Estimate the behavior of dynamic systems. !!
Apply of decision-making concepts, including neural networks, expert
systems, and genetic algorithms.!
Components of systems for decision-making and control, such as
sensors, actuators, and computers. !!
Systems-engineering approach to the analysis, design, and testing of
robotic devices.
!!
Computational problem-solving, through thorough knowledge,
application, and development of analytical software. !!
Historical context within which robotics and intelligent systems have
evolved.!
Global and ethical impact of robotics and intelligent systems in the
context of contemporary society.
!!
Oral and written presentation.!
55