School of Aeronautics and Astronautics
Aircraft Conflict Resolution:
Aircraft Conflict Resolution: A Stochastic Optimal Control Approach
Inseok Hwang
(with Wei Liu)
Fli ht D
Flight Dynamics and Control/Hybrid System Laboratory
i
d C t l/H b id S t
L b t
School of Aeronautics and Astronautics
Purdue University
Workshop for Aerospace Decision and Control, Georgia Tech
Atlanta, GA , June 12, 2012
Air Traffic to and from Atlanta Airport
movie
Outline
• Background and motivations
k
d d
• Aircraft
Aircraft dynamics and wind dynamics: Stochastic Differential dynamics and wind dynamics: Stochastic Differential
Equations (SDEs)
• Problem formulation: Stochastic Optimal Control
• Solving the optimal control problem using approximating Markov Chain.
• Simulations
Background
• Current Air Traffic Control (ATC)
– Centralized control by ground ATC systems
– Rigid and hierarchical structure
– Not well scaled up to ever increasing air traffic demand
• Next Generation Air Transportation System (NextGen)
– Collaborative
Collaborative decision making: flat, net decision making: flat net
centric control
– Aircraft are allowed to coordinate with each other
– Safety enhancement by safe separation assurance
•
Conflict Detection and Resolution
(
(CD&R)
)
[1] FACET: NASA national aerospace modeling and simulation software
[2] http://innopedia.wikidot.com/nextgen‐defined
Aerospace Related Research Topics
• Air Traffic Control
Air Traffic Control
– Air traffic surveillance
• Multi‐aircraft tracking and identity management
• Multi‐sensor data fusion and fault detection
–
–
–
–
–
–
Separation assurance: conflict detection and resolution
Trajectory based operation: trajectory prediction and ETA computation
Safety through conformance monitoring (airspace and airport) in ATC
y
g
g( p
p )
Dynamic Airspace Configuration (en‐route and terminal airspace)
Arrival flight scheduling and sequencing
Performance metrics for NAS
• UAS applications
– Modeling and simulations using open source environment (Mixed Airspace)
– Autonomous navigation and control of UAS (e.g., sense
Autonomous navigation and control of UAS (e g sense‐and
and avoid)
avoid)
– Cyber‐secure UAS design and control
• Other applications
– Mode
Mode awareness (mode confusion) using FDIR for (Stochastic) Hybrid Systems
awareness (mode confusion) using FDIR for (Stochastic) Hybrid Systems
– Space applications: non‐Keplerian spacecraft tracking and low‐thrust orbit transfer
Aerospace Related Research Topics
• Air Traffic Control
Air Traffic Control
– Air traffic surveillance
• Multi‐aircraft tracking and identity management
• Multi‐sensor data fusion and fault detection
–
–
–
–
–
–
Separation assurance: conflict detection and resolution
Trajectory based operation: trajectory prediction and ETA computation
Safety through conformance monitoring (airspace and airport) in ATC
y
g
g( p
p )
Dynamic Airspace Configuration (en‐route and terminal airspace)
Arrival flight scheduling and sequencing
Performance metrics for NAS
• UAS applications
– Modeling and simulations using open source environment (Mixed Airspace)
– Autonomous navigation and control of UAS (e.g., sense
Autonomous navigation and control of UAS (e g sense‐and
and avoid)
avoid)
– Cyber‐secure UAS design and control
• Other applications
– Mode
Mode awareness (mode confusion) using intent inference
awareness (mode confusion) using intent inference
– Space applications: non‐Keplerian spacecraft tracking and low‐thrust orbit transfer
Aircraft Conflict Resolution: Overview
• Two
Two kinds of midair conflict: Aircraft
kinds of midair conflict: Aircraft‐weather
weather and Aircraft
and Aircraft‐aircraft
aircraft conflict
• Most conflict resolution algorithms are to d
fl
l
l
h
design conflict‐free trajectories for aircraft to follow (open‐loop control)
– Easy to understand and compute
– Easy to implement
– Could be inefficient and/or unsafe if aircraft deviate from the optimal
if aircraft deviate from the optimal Conflict‐free trajectories
Closed‐loop Optimal Control Algorithm
[1] Figure source: THALES
Proposed Aircraft Conflict Resolution
• We
We propose to use a closed‐loop
propose to use a closed loop technique to solve this problem which technique to solve this problem which
can explicitly account for uncertainties (such as wind)
Cockpit display of Traffic Collision Avoidance System (TCAS)
Nearby aircraft positions
Suggested climb gg
rate
Stochastic System Dynamics
• Aircraft and Wind dynamics in the horizontal plane
Aircraft and Wind dynamics in the horizontal plane
– Aircraft:
: aircraft position; : aircraft heading (control input); : cruise speed; : wind velocity vector
– Wind:
: deterministic wind velocity; : intensity of uncertainties in wind dynamics; : Brownian motion
y
;
• Combined Dynamics (Stochastic Diff. Eq.)
Deterministic wind field
Aircraft‐Weather Conflict Resolution
• Aircraft‐weather conflict resolution as an optimal control problem
Aircraft weather conflict resolution as an optimal control problem
– Given aircraft and wind dynamics
– We want to drive the aircraft to the target set without entering the
target set without entering the forbidden or exit the whole area.
– Cost
1
Running cost
Exit cost
where is a stopping time
Design to minimize the cost , subject
to the aircraft and wind dynamics
r(x) takes high value on boundaries of the forbidden sets and low value on boundary of the
and low value on boundary of the target set.
Aircraft dynamics
Feedback control
Feedback
Stochastic Optimal Control Problem
• Optimal control problem
O ti l
t l
bl
– Dynamics:
– Minimize:
• Hamilton‐Jacobi‐Bellman (HJB) equation
– Cost‐to‐go: starting from a point ,
Cost to go: starting from a point
– HJB equation:
Approximating Markov Chain
• SSolving the stochastic optimal control: a Markov Chain (MC) l i
h
h i
i l
l M k Ch i (MC)
approach
– Discretize the state space using grids:
– Design the transition probabilities – If the convergence relations hold, the MC converges to the SDE
N(q) : Neighbors of q
Jacobi Iteration for Optimal Control of Markov Chain
– Optimal control problem: • Minimize: Cost-to-go
C
tt
off th
the
Markov chain
First time when the Markov
chain exit the domain
Running cost
– HJB equation (discrete version):
Exit cost
This HJB can be solved numerically!
– Jacobi iteration (similar to gradient descent):
J bi it ti ( i il t
di t d
t)
Convergence relation
Numerical Simulations
• Aircraft‐weather avoidance scenario
Ai
ft
th
id
i
Cost‐to‐go computed by Jacobi iteration
Optimal control input (aircraft heading) and 50 aircraft trajectories
Aircraft‐Aircraft Conflict Resolution
• Two‐aircraft dynamical model
• Forbidden set:
• Optimal control problem: Optimal control problem:
– Choose to minimize:
–
takes high value on the boundary of Aircraft‐Aircraft Conflict Resolution
Numerical Simulation Results
• Optimized cost‐to‐go and optimal control
Optimized cost to go and optimal control
input to Aircraft 1 when Aircraft 2 is in different
locations
When Aircraft 2 is outside S0, Aircraft 1’s y
control is not affected by Aircraft 2.
When Aircraft 2 is inside S0, Aircraft 1’s control is affected by Aircraft 2
control is affected by Aircraft 2.
Numerical Simulation Results
• Optimal cost function and optimal control input
Optimal cost function and optimal control input
Multi‐aircraft Conflict Resolution
• 4‐aircraft conflict resolution scenario
f
fl
l
Computational Complexity Analysis
• The computational complexity of the proposed algorithm increases h
l
l
f h
d l
h
rapidly as the number of aircraft involved increases (curse of dimensionality)
• To overcome the rapid glowing computation time, a decomposition method is proposed
method is proposed
• Comparison of the computation time (sec) of the conflict resolution algorithm with and without decomposition (using MATLAB)
Summary
• The aircraft and wind dynamics are modeled by Stochastic Differential Equations (SDE)
• The conflict resolution problem is formulated as the optimal control of SDE
• To solve the optimal control problem, a Markov chain is constructed to approximate the SDE
approximate the SDE
• The performance of the proposed conflict resolution algorithm has been demonstrated with illustrative conflict scenarios
• Future work: – Using Differential Transformation to further improve computational complexity to better accommodate high fidelity models and complex scenarios I.
II.
Hwang, J. Li, and D. Du, “Differential Transformation and Its Application to Nonlinear Optimal Control,” ASME Journal of Dynamic Systems, Measurement, and Control, Vol. 131 (5), September 2009
R. Huang, I. Hwang, and M. Corless "A Nonlinear Algorithm for Tracking Interplanetary Low‐Thrust Trajectories," AIAA Journal of Guidance, Control and Dynamics, To appear