6 Degree(V)=4 Unseeded PENBMI

Development of Analysis Tools for Certification of Flight
Control Laws
FA9550-05-1-0266, April 05-November 06
Participants
UCB: Ufuk Topcu, Weehong Tan, Tim Wheeler, Andy Packard
Honeywell: Pete Seiler
UMN: Gary Balas
Website
http://jagger.me.berkeley.edu/~pack/certify
Copyright 2006, Packard, Tan, Wheeler, Seiler and Balas. This work is licensed under the Creative Commons Attribution-ShareAlike License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott
Way, Stanford, California 94305, USA.
Tools for Quantitative, Local Nonlinear Analysis
Focus over the past 15 months
– Region of attraction estimation
– L2  L2 induced norms
– L2  L induced norms
for
– Locally stable, finite-dimensional nonlinear systems, with
• polynomial vector fields
• parameter uncertainty (also polynomial)
Main Tools:
– Lyapunov/HJI formulation
– Sum-of-squares proofs to ensure nonnegativity and set containment
– Semidefinite programming (SDP), Bilinear Matrix Inequalities (BMI)
• Optimization interface: YALMIP and SOSTOOLS
• SDP solvers: Sedumi
• BMIs: using PENBMI (academic license from www.penopt.com)
– Constraints provided by simulation
Examples presented last year
w
C ˆ

u

P
 
x2
x1
0.5
L2 to L2 gain
x1  x2   ( x1  1)
x2  u

 
Γ=1
0.45

G 1ˆ  x1 x1  1  x2  x1  ˆx1  1 1  ˆ x1  1
u   x2  x1  ˆx1  1  x1  1  ˆ x2  ˆ x1  1

 ˆx1  1

Adaptive Control,
G = 1 and
G= 4
0.55

0.4
0.35
0.3
0.25
0
Vi q  1 q  2
2 13
38
4
57 120
6 166 338
max
w 2 R
Γ=4
0.5
1
x1
2
w2
1.5
R


16


x1   x1 1  x22  x2
14 x   x 1  x 2  w
2
2
1
12
Composite (2 Vi)
10
b
p( x)  Cx
Upper Bound
2
8 C   8  4
  4 4


6
Refined Upper Bound
1
2
Lower Bnd
Linearized
4
2
0
0
2
4
Using worst-case
input from linear
analysis
6
8
10
2
R
2
Sum-of-Squares
Sum-of-squares decompositions will be the main tool to
decide set containment conditions, and certify nonnegativity.
A polynomial f, in n real-variables is a sum-of-squares if it
can be expressed as a sum-of-squares of other polys,
p
f   g 2j
j 1
 n denotes set of all sum-of-square polynomials in n
variables
Sum-of-Squares Decomposition
For a polynomial f, in n real-variables, and of degree 2d
f  n

M 0 such that f  z T Mz
where z  [1, x1 , x2 ,, xn , x1 x2 ,, xnd ]T .
The entries of z are not algebraically independent
– e.g. x12x22 = (x1x2)2 )
– M is not unique (for a specified f)
The set of matrices, M, which yield f, is an affine subspace
– one particular + all homogeneous
– Particular solution depends on f
– all homogeneous solutions depend only on n & d.
Searching this affine subspace for a p.s.d element is an SDP…
Sum-of-Squares as SDP
For a polynomial f, in n real-variables, and of degree 2d
f  n

q
  R q such that M 0   i M i  0
i 1
Semidefinite program: feasibility
Each Mi is s×s, where
n  d 
s

 d 
2

n

d
1 
  n  d    n  2d 
q  
 
  

2  d   d   2d 
Using the Newton polytope method, both s and q can
often be reduced, depending on the terms present in f.
(s,q) wrt n and 2d
n  d 

s
 d 


2d
n
 n  d  2  n  d   n  2d 
1 
 
  

q
2  d   d   2d 


2
4
6
8
2
3
0
6
6
10
27
15
75
3
4
0
10
20
20
126
35
465
4
5
0
15
50
35
420
70
1990
6
7
0
28
196
84
2646
210
19152
8
9
0
45
540 165 10692
495
109890
10
11
0
66 1210 286 33033 1001
457743
Synthesizing Sum-of-Squares as SDP
Given: polynomials f 0 , f1 , , f m
Decide if an affine combination of them can be made a
sum-of-squares.
This is also an SDP.
m
  R m with f 0   k f k   n
k 1

qm
  R q  m with M 0   i M i  0
i 1
Synthesizing Sum-of-Squares as Bilinear SDP
Given: polynomials
f 0 , f1 ,  , f m
g 0 , g1 , , g m
h0 , h1 ,  , hm
A problem that will arise in this talk is: find
such that
  R m ,  R m
m
m



f 0   k f k   g 0   k g k  h0   k hk    n
k 1
k 1
k 1



m
This is a nonconvex SDP, namely a bilinear matrix inequality
Common features of analysis
These analysis all involve search over a nonconvex set of
certifying Lyapunov functions, roughly
x : x  0,V ( x)  1  x : V  f
 0
The SOS relaxations are nonconvex as well, e.g.,
find V ,V (0)  0, and s8 , s9   n
subject to
 1  V s8  s9V  f  l2    n
They are “solved” via
– PENBMI, commercial BMI solver from PENOPT
– Ad-hoc iteration on linear SDPs
– Examples were nonconvex problems in ~100s of variables
Last year: What we didn’t show…
Obtaining results was challenging….
restart
Run PENBMI
diverge
Answer not what is
needed or expected
restart
Run Iteration
YES!
YES!
Answer not what is
needed or expected
By contrast:
– Today’s results better, reliable and naturally obtained
For now
– Restrict attention to region-of-attraction estimation
Estimating Region of Attraction
Nonconvex problem, nonconvex relaxation.
Solution approaches: SOS conditions to verify containments
– Parametrize V, parametrize multipliers, solve…
• Bilinear SDP solvers
• Ad-hoc iterative, based on linear SDPs
Behavior:
– Initial point has big effect on end result, e.g.,
• Unable to reach a feasible point
• Convergence to local optimum
What are prospects for generating “good” initial points?
– Easily computable
– Promising results
Estimating Region of Attraction
Dynamics, equilibrium point
x  f ( x),
f (x)  0
V 1
p3
User-defined function whose
sub-level sets are to be in
region-of-attraction
x : p( x)  b   ROA x
p2
p 1
x
By choice of positive-definite
V, maximize b so that
x:V(x) 1 is bounded
x : p( x)  b   x:V(x) 1
 dV
x : x  x ,V ( x)  1   x:


f  0
dx

dV
f 0
dx
Region of Attraction: Bilinear SOS
V 1
p3
p2
Maximize b (positive-definite V ) so that
x : V(x)  1 is bounded
x : p( x)  b   x : V(x)  1
x : x  x,V ( x)  1  x : V  f
p 1
x
 0
Choose “small” positive definite functions
dV
f 0
dx
l1 ,l2
max b over V , V ( x )  0, s6 , s8 , s9   n
subject to
V  l1   n
BMIs
 b  p s6  V  1   n
 1  V s8  s9V  f  l2    n
Products of
decision variables
Sanity check
For a positive definite matrix B,
x   x  xBx xROA  x : xBx  1
Proof: V x  : xBx
0
 V  2V V  1.
nth order system
cubic vector field
known ROA
Consider p.d. quadratic shape factor p( x)  xT Rx
The best obtainable result is the “largest” value bmax such that
x : xRx  b   x : xBx  1
That containment easy to characterize:
 
 12
bmax  max R BR
 12
Questions:
Yes
– Can the bilinear SOS formulation yield this?
– Can the BMI solver find this solution? Basically, Yes, Fast (n<10)
1000’s of random examples, n=2-10; two
restarts of PENBMI, always successful

1
Region of Attraction
Consider a simpler question. Fix β, is
Pb : x : p( x)  b   ROA x ?
Ad-hoc solution:
– run N sims, starting from samples in
x : p( x)  b 
• If any diverge, then “no”
• If all converge, then maybe “yes”, and perhaps the Lyapunov analysis
can prove it
In this case, how can we use the simulation data?
Necessary condition: If V exists to verify, it must be
– ≤1 on all trajectories
– ≥0 on all trajectories
– Decreasing on all trajectories
– and possibly some more…
x : p( x)  b   x:V(x)  1bounded
x : x  x ,V ( x)  1   x: dV


f  0
dx

Outer bound on certifying Lyapunov functions
After simulations
– Collection of convergent trajectories starting in
c
– divergent trajectories starting in Pb
Linearly parametrize V, namely
Pb
Nb
V ( x )    k k ( x )
k 1
The necessary conditions on V are convex constraints on   R N b
V≤1 on convergent trajectories
V≥0 on all trajectories
V ( x)  l1 ( x)   n ( x)
V decreasing on convergent trajectories
Quad(V) is a Lyapunov function for Linear(f)
V≥1 on divergent trajectories
Hit & Run: Uniformly sample convex set in Rn
1.
2.
3.
4.
5.
Start with an interior point, w
Pick a direction v in Rn, N(0,I)
Find tmin and tmax such that w+tv just in set
Pick μ, uniformly in [tmin tmax]
NextX = x + μv
In Lyapunov coefficient space, get samples:
– Assess the ROA that they certify, or…
– Use as a seed for
•
PENBMI, and/or iteration
Finding [tmin tmax] involves
– Several simple 1-d linear inequalities
– A linear matrix inequality for
AT P  PA  0
– An SOS program, for
V  l1   n
Smith, 1984 Operations Research
Lovasz, 1999 Math Programming
Tempo, Calafiore, Dabbene, Springer
Assessing V: Checking containments
Each candidate V certifies a region of attraction
b cert ,V : max b such that  satisfying
x : p( x)  b   x : V ( x)     x : V ( x)  0
Generally, this is solved in two steps
– SOS optimization (s8, s9) to maximize the level-set condition on V
   V s8  s9V  f  l2   n
– SOS optimization (s6) to maximize the condition on p & V
 b  p s6  V     n
PENBMI and iteration initialized with these as well
Assessing V: Checking containments
Alternate conditions, this is solved in two steps
– SOS optimization (p1) to maximize the level-set condition on V
V   i1 xi2d
n
1
 p1V  f   n
SDP, no bisection
– SOS optimization (p2) to maximize the condition on p & V
 b  p  x
n
2d2
i 1 i
Under the assumption that
0, these confirm
 V    p2   n
SDP, no bisection
V  f is negative definite near
b cert ,V : max b such that  satisfying
x : p( x)  b   x : x  0,V ( x)   cc ,0  x : V ( x)  0
Employing simulation
Simulate
Sample Vouter
Assess ROA using V
Seed Iteration
Seed PENBMI
Examples considered
vanDerPol’s
x1   x2
x2  x1  x12  1x2
Limit cycle for VDP
3
2
x2
1
0
-1
-2
-3
-2.5
-2
-1.5
-1
-0.5
0
x1
0.5
1
1.5
2
2.5
Examples considered (cont’d)
Aircraft: Pitch axis, 2-state dynamic inversion controller
 x1 x2  x22  x2 x3  x2 x4  x2 x5  x23 


2
x2 x5  x5


x  Ax  
0



0


0


q
 
 
x  1 
 
 2
  



A  





0
0
0



0
0



0
0






0
Short period
longitudinal model,
Results: Van der Pol’s oscillator
Quadratic shape factor: βmax ≈1.04
Sims performed from x : p( x)  0.6
Assess achieved β from 50 samples of outer bound


th
4nd
26
order V
V
order
12
number of points
10
8
6
4
2
0
0.3
0.4
0.5
b LP
LP
Now seed PENBMI with these samples
0.6
0.7
0.8
Van der Pol’s Summary
Unseeded PENBMI
Degree(V)=4
Degree(V)=6
RunTime
30-45(-300) seconds
900-3000 seconds
BestAnswer, β=
0.928
1.034
Percentage
90
30
Seeded PENBMI
Degree(V)=4
Degree(V)=6
Simulations
10 seconds (100)
20 seconds (200)
Form LP/ConvexP
1 second
2 seconds
Get feasible point
10 seconds
20 seconds
Associate multipliers
2 seconds
5 seconds
Seed/Run PENBMI
7 seconds
16 seconds
TOTAL
35 seconds
63 seconds
Additional Point (H&R)
0.1 seconds
0.2 seconds
Associate multipliers
2 seconds
5 seconds
Seed/Run PENBMI
7 seconds
16 seconds
BestAnswer, β=
0.930
1.034
Percentage
100
100
Level Sets
x : V ( x)  1
Level set of V for VDP
3
2
1
x2
The level sets
0
-1
nV = 2
nV = 4
-2
n =6
V
-3
-2.5
-2
-1.5
-1
-0.5
0
x1
0.5
1
1.5
2
2.5
What did the aircraft analysis entail/yield/require?
Several Analysis
max b over V , V ( x )  0, s6 , s8 , s9   n
– Unseeded calls to PENBMI
subject to
V  l1   n
– Sim-based initialization for iteration
 b  p s6  V  1   n
• Did not run a seeded PENBMI
 1  V s8  s9V  f  l2    n
– Alternate initialization for iteration
– Separate extensive simulations to find divergent trajectories
Simple form for shape factor
p( x) : xT x
Different Lyapunov function structures
– Quadratic (8.6, 8.6)
– pointwise-max quadratics (8.6)
– Quadratic+Quartic (12.2, 12.2)
– Fully quartic (quadratic + cubic + quartic)
Results: quadratic+cubic+quartic V
There is a divergent trajectory starting from p( x0 )  16.1
Simulation-based algorithm
divergent trajectory from
4000 simulations
5 minutes
Form LP/ConvexP
3 minutes
Get a feasable point
5 minutes
Assess answer with V
2 minutes
Iterate from V
3 minutes/iteration, 6 iters
TOTAL
33 minutes
p( x0 )  16.9
x : p( x)  14.6  ROA
Iteration from “random” starting point
AT P  PA   I
5
T
V ( x)  x Px  0.001i 1 xi4
Take P from
30 iterations
x : p( x)  8.5  ROA
Direct unseeded call to PENBMI yields (after 38 hours)
– All initial conditions in
x : p( x)  15.2 are in ROA
What’s possible?
Assuming no breakthroughs in
– SDP/BMI solvers
– exploiting problem structure
Then, reliable and time-tolerable analysis for systems with
– Cubic vector fields
– State dimension between 10 and 15, pointwise-max quadratic
Lyapunov functions
– State dimension ≤6, quartic Lyapunov functions
How should this be viewed?
– Linearized analysis is effectively
• Infinitesimal analysis of dynamics with quadratic Lyapunov fcns
– So, the proposed method extends both the degree of approximation
of the dynamics, and the richness of the Lyapunov function
Extensions
Using simulation data to impose necessary conditions on
Lyapunov/storage functions that prove
– Local, input/output gain bounds
– Local state reachability
– Attractive invariant sets
– ROA for uncertain systems
x  f ( x, w), z  h( x)
extends (conceptually) easily.
If V  f x, w  wT w   2 z T z on V x   R 2 ; V (0)  0, V  0
Then x(0)  0
 V ( x(t ))  R 2 , z 2   R
w2R
If such a V exists, then from x(0)=0, with w 2  R there are
– upper and lower bounds on V, and
– upper bounds on V
Some things to pursue
Principal component analysis on the Lyapunov coefficient
space (manifold discovery, Coifman)
– Superficially, this won’t help much, since most of the variables in the
SOS optimization come from the nonuniqueness of the Gram matrix,
and are a function of the degree and order.
McEnneay’s curse-of-dimensionality free computing
More inner-loop airframe closed-loop analysis
Wang, Lall, West (2005 Allerton) level set advection methods
– Approximately integrate polynomial level set backwards, preserving
polynomial structure. Iteration of linear SDPs
Effect of number of simulations, employing backward sims
Alternate iterations
Time delays, other robustness metrics
Problems, difficulties, risks
Dimensionality:
– Theory leads to reduced complexity in specific instances of
problems (sparsity, Newton polytope reduction, symmetries)
Solvers (SDP): numerical accuracy, conditioning
Connecting the input/output gains to other measures of
robustness and performance
– Decay rates
– Damping ratios
– Oscillation frequencies
– Including time-delays
BMI nature of local analysis, though the simulation-based
approach to seed BMI is promising