Data-Driven Behavioral
Formulation of the
Adaptive Feedback
Control Problem
Michael G. Safonov
University of Southern California
AFOSR Dynamics and Control Meeting, WPAFB
1
Michael G. Safonov 30 July - 2 August 2001
An Achilles heel of modern
system theory has been the
habit of
‘proof by assumption’
Theorists typically give insufficient attention to the
possibility of future observations which may be at odds
with assumptions.
AFOSR Dynamics and Control Meeting, WPAFB
2
Michael G. Safonov 30 July - 2 August 2001
Controller Adaptation
should be intelligent.
• Intelligence is consistency between FACTS,
DECISIONS, & GOALS.
DECISIONS
controller,
estimator
FACTS
assumptions,
beliefs, models
or data
AFOSR Dynamics and Control Meeting, WPAFB
GOALS
cost functions,
objectives
3
Michael G. Safonov 30 July - 2 August 2001
1.
But, there are two
kinds of FACTS
Observed DATA
(data/evidence)
2. Prior BELIEFS
(assumptions/models/axioms)
Science gives DATA precedence over BELIEFS:
BELIEFS inconsistent with DATA are rejected.
AFOSR Dynamics and Control Meeting, WPAFB
4
Michael G. Safonov 30 July - 2 August 2001
And two ways to learn & adapt:
Observation vs. Introspection
Reality is an ideal, observable only
through noisy sensors.
Reality is what we observe.
vs.
Galileo: open-eyed
“data-driven”
MODELS approximate Observed Data
Plato: introspective
“assumption driven”
Data approximates Unobserved TRUTH
‘Curve-Fitting’
AFOSR Dynamics and Control Meeting, WPAFB
‘Probabilistic Estimation’
5
Michael G. Safonov 30 July - 2 August 2001
Example: Linear Regression
BAYESIAN
ESTIMATE (Platonic):
Given data (yi,ui), i=1,2,...
max
(y1,u1)
prob (y |x, a , b )
subject to prior beliefs
y  au  b  v
(y7,u7)
(y2,u2)
(y4,u4)
‘noise’ v=N(0, )
(y5,u5)
(y3,u3)
(y6,u6) CURVE FIT: (Galilean)
Given data (yi,ui), i=1,2,...
height 2
min
{
AFOSR Dynamics and Control Meeting, WPAFB
|| y  au  b ||
i
i
i
6
Michael G. Safonov 30 July - 2 August 2001
1
2
Both the Bayesian probabilist and the Galilean
curve-fitter use the same formula to estimate model
parameters (a,b), but a naive Bayesian may have
some (rather unrealistic) expectations for his model.
(y1,u1)
The assumption driven
Bayesian ‘knows’ a priori
(y7,u7)
that 2/3 of his future data (y ,u )
4 4
(y2,u2)
must eventually lie in his
predicted 2 confidence
(y3,u3)
bound, and 1/3 outside
The data driven Galilean
(y5,u5) (y6,u6) curve fitter will remain
open minded: He will wait
to look and see how his
model fits the data.
height 2
{
AFOSR Dynamics and Control Meeting, WPAFB
7
Michael G. Safonov 30 July - 2 August 2001
And, Galilean vs Bayesian:
two kinds of adaptive control
“Data Driven”
•
•
“Assumption Driven”
goals are modest:
• goals are ambitious:
– no guaranteed predictions
– guaranteed future stability
of future stability
– Cost & estimation convergence
– just consistency of goals,
• many troublesome assumptions
decisions and data
no troublesome assumptions, – “the ‘true’ plant is in a given model set”,
parsimonious formulation:
– “noise independent identically
– DATA
– GOALS
– DECISIONS
• no approximation
distributed”
– “bounds on parameters, probabilities”
– …, “linear time-invariant, minimumphase plant, order < N”
Remarkably, some leading “assumption driven” control theorists have held that
observed DATA inconsistent with ASSUMPTIONS should be ignored (cf. M.
Gevers et al., "Model Validation in Closed-Loop", ACC, San Diego, 1999)
AFOSR Dynamics and Control Meeting, WPAFB
8
Michael G. Safonov 30 July - 2 August 2001
Data Driven Adaptive Control
ACTIVE
CONTROLLER
SELECTOR
K
z  u, y
candidate
controllers
DATA
given
GOALS
COMPUTER
SIEVE
LEARNING FEEDBACK LOOPS
DECISIONS
Unfalsified
Controllers K
FALSIFIED
M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996.
AFOSR Dynamics and Control Meeting, WPAFB
9
Michael G. Safonov 30 July - 2 August 2001
The Behavioral Approach
to Adaptive Control
• Data Driven: Let the data speak...
– Don’t let modeling beliefs trump observation
• Unfalsify (validate) models and/or
controllers against hard criteria:
– Choose criteria expressible directly in terms of
observed data (sensor outputs, actuator inputs)
– Avoid criteria that that rely on “noise model”
and other prior beliefs
AFOSR Dynamics and Control Meeting, WPAFB
10
Michael G. Safonov 30 July - 2 August 2001
t
i Unfalsification
Algorithm
o
UNFALSIFIED
ADAPTIVE
n
CONTROL:
A
The ability of each
candidate controller tol
meet the performance
goal is treated as a g
hypothesis to be tested
o
directly against
evolving real-time r
measurement data.
The controller need not
i
be in the loop to test
t
the hypothesis.
h
11
AFOSR Dynamics and Control Meeting, WPAFB
Brugarolas & Safonov, CCA/CACSD ‘99
Michael G. Safonov 30 July - 2 August 2001
Trivial Example
r
+

-
e
u
K
Unknown
Plant
gain
• Plant Data:
at time t=0, (u,y)=(1,1)
• Candidate K’s: u=Ke real gain
• Goal:
|e(t)/r(t)| < 0.1 for all r(t)
Relations: e=u/K=1/K , r=y+e=y+u/K= 1+1/K
=> K is unfalsified if |1/(1+K)| < 0.1
=> unfalsified K’s: K>9 or K<-11
AFOSR Dynamics and Control Meeting, WPAFB
12
Michael G. Safonov 30 July - 2 August 2001
y
DATA-DRIVEN
Behavioral Problem Formulation
• Observations operator P maps input-output signals to
measurement signals.

• Truncated Space Z  P Z results from applying the
observations operator to a signal space.
Typically P is the experimental observation time
sampling operator, so P z returns values of z(t) only for
past time intervals over which experimental observations
of z(t) have been recorded. P is a projection operator.
AFOSR Dynamics and Control Meeting, WPAFB
13
Michael G. Safonov 30 July - 2 August 2001
BACKGROUND: Willems’
Behavioral System Theory
AFOSR Dynamics and Control Meeting, WPAFB
14
Michael G. Safonov 30 July - 2 August 2001
BEHAVIORAL
ADAPTIVE CONTROL
AFOSR Dynamics and Control Meeting, WPAFB
15
Michael G. Safonov 30 July - 2 August 2001
Application:
Behavioral MRAC
Behavioral Representation of Standard
Class of Candidate MRAC Controllers
AFOSR Dynamics and Control Meeting, WPAFB
16
Michael G. Safonov 30 July - 2 August 2001
Behavioral MRAC (cont.)
AFOSR Dynamics and Control Meeting, WPAFB
17
Michael G. Safonov 30 July - 2 August 2001
Main Result: Solution of
Behavioral MRAC Problem
A matrix pencil e-value computation gives optimal
AFOSR Dynamics and Control Meeting, WPAFB
18
Michael G. Safonov 30 July - 2 August 2001
.
Data Driven Behavioral
MRAC Computation
AFOSR Dynamics and Control Meeting, WPAFB
19
Michael G. Safonov 30 July - 2 August 2001
Noisy Plant
Simulation Example
AFOSR Dynamics and Control Meeting, WPAFB
20
Michael G. Safonov 30 July - 2 August 2001
Data Driven Behavioral
MRAC Simulation Results
AFOSR Dynamics and Control Meeting, WPAFB
21
Michael G. Safonov 30 July - 2 August 2001
Discussion: Data Driven
Behavioral MRAC
•
•
•
•
Unstable plant stabilized in real time
Computation effort does not grow with 
No need for controller parameter gridding
Exponential forgetting factor in cost
– no controller ‘gain windup’
– behaves like classical s- modification
AFOSR Dynamics and Control Meeting, WPAFB
22
Michael G. Safonov 30 July - 2 August 2001
Other Examples:
Unfalsified PID Control
• Unfalsified adaptive control loop stabilizes in real-time
• Unstable Plant 30 Candidate PID
Controllers:
Indices of Unfalsified Controller
30
Index Current Active Controller
25
KI =[2, 50, 100]
KD = [.5, .6]
KP = [5, 10, 25, 80, 110]
Example: Adaptive PID
Index of K
20
6
10
Command R
Control U
Output Y
4
5
Goal:
2
w1 * (r  y )   w2 * u
2
where f
15
2


2

 s 2  r
2


0
0
0
2
  f (t ) dt
5
10
Time (second)
15
20
5
10
time
15
20
0
-2
-4
-6
-8
-10
—————
-12
Jun & Safonov, CCA/CACSD ‘99
AFOSR Dynamics and Control Meeting, WPAFB
23
0
Michael G. Safonov 30 July - 2 August 2001
Benchmark
Simulation
control input u(t)
plant output y(t)
proportional gain kP (t)
integral gain kI (t)
derivative gain kD (t),
Jun & Safonov, CCA/CACSD ‘99
Tsao & Safonov, IEEE Trans, AC-42, 1997.
Evolution of unfalsified set
AFOSR Dynamics and Control Meeting, WPAFB
24
Time responses
Michael G. Safonov 30 July - 2 August 2001
Another Example: Missile
• Learns control gains
• Adapts quickly to
compensate for
damage & failures
• Superior performance
Specified target response bound
Actual response
Brugarolas, Fromion and Safonov, ACC98
Unfalsified adaptive missile autopilot:
• discovers stabilizing control gains
as it flies, nearly instantaneously
• maintains precise sure-footed control
AFOSR Dynamics and Control Meeting, WPAFB
25
Commanded
response
Brugarolas, Fromion & Safonov, ACC ’98
Michael G. Safonov 30 July - 2 August 2001
Other People’s Successes
with Unfalsified Control
• Emmanuel Collins et al. (Weigh Belt Feeder
adaptive PID tuning, CDC99)
• Kosut (Semiconductor Mfg. Process run-torun tuning, CDC98)
• Woodley, How & Kosut (ECP Torsional
disk control, adaptive tuning, ACC99
• … maybe others soon?
AFOSR Dynamics and Control Meeting, WPAFB
26
Michael G. Safonov 30 July - 2 August 2001
Conclusion
• We have analytic tools for
controlling assumed models
• DATA-DRIVEN analytic tools
needed to reliably close
adaptive feedback loops with
experimentally observed data
A SOLUTION:
DATA-DRIVEN
Unfalsified Adaptive
AFOSR Dynamics and Control Meeting, WPAFB
27
Michael G. Safonov 30 July - 2 August 2001
Acknowledgement
Bob Kosut’s mid-1980’s work on time-domain model validation and
identification for control played a key role in laying the foundations of this
work, as did later contributions of Jim Krause, Pramod Khargonekar,
Carl Nett, Kamashwar Poolla, Roy Smith and many others who have
advanced the use of validation methods in control-oriented identification.
Tom Mitchell’s early 1980’s “candidate elimination algorithm’’ for
machine learning is closely related to the unfalsified control methods
presented here. And of course, none of this would have been possible
without the superb graduate education that I received at MIT so many years
ago under the guidance of first Jan Willems and later Michael Athans.
AFOSR Dynamics and Control Meeting, WPAFB
28
Michael G. Safonov 30 July - 2 August 2001
Selected References
AFOSR Dynamics and Control Meeting, WPAFB
29
Michael G. Safonov 30 July - 2 August 2001