Bi-Parametric Convex
Quadratic Optimization
Tamás Terlaky
Lehigh University
Joint work with Alireza Ghaffari-Hadigheh
and Oleksandr Romanko
RUTCOR 2009: Dedicated to the 80th Birthday
of Professor András Prékopa
2
Outline
Introduction
 Quadratic optimization, optimal partition
 Uni-Parametric quadratic optimization
 Bi-Parametric quadratic optimization
 Numerical illustration
 Fundamental properties
 Algorithm
 Conclusions and future work

3
Introduction: Parametric Optimization
General framework of parametric optimization

Multidimensional parameter is introduced
into objective function and/or constraints

The goal is to find
•
•


– optimal solution
– optimal value function
Generalization of
sensitivity analysis
Applications:
multi-objective
optimization
4
Introduction: Multi-Objective
Optimization as Parametric Problem

Multi-objective optimization:
OBJECTIVE SPACE

f2
Multi-objective optimization with weighting method:
identify Pareto
frontier (all nondominated solutions)
f*2

Parametric formulation:
f1
f*1
5
Introduction: Quadratic Optimization
and Its Parametric Counterpart

Convex Quadratic Optimization (QO) problem:

Bi-Parametric Convex Quadratic Optimization (PQO) problem:

Bi-parametric QO generalizes three models:
uni-parametric QO
6
Sensity Analysis: Just be careful!
7
Optimal Partition for QO
Convex Quadratic Optimization problems:

Primal

Optimality conditions:

Maximally complementary solution:
•
•
LO:
QO:
IPMs !!!
Dual
and
- strictly complementary solution
, but
may not hold
maximally complementary solution maximizes the number
of non-zero coordinates in and
8
Optimal Partition for QO
of the index set {1, 2,…, n} is

The optimal partition

The optimal partition is unique!!!
An optimal solution
is maximally complementary iff:

Example: for maximally complementary solution
5
3
0
0
0
0
0
0
0
1.08
with:
9
Uni-Parametric Quadratic Optimization

Primal and dual perturbed problems:

For some
solution
partition

The left and right extreme points of the invariancy interval:
we are given the maximally complementary optimal
of
and
with the optimal
.
- invariancy interval
- transition points
10
Gengyang
Uni-Parametric QO: Optimal Partition in
the Neighboring Invariancy Interval
Solve two auxiliary
problems

How to proceed from the current invariancy interval to the next
one?
(1)
(2)
z
z
z
11
Uni-param QO:
Numerical
Illustration
Solver output
type
l
u
B
N
T
()
----------------------------------------------------------------------------------------------------------------transition point -8.00000 -8.00000
3 5
1 4
2
-0.00
invariancy interval -8.00000 -5.00000
2 3 5
1 4
8.502 + 68.00 + 0.00
transition point -5.00000 -5.00000
2
1 3 4 5
-127.50
invariancy interval -5.00000 +0.00000
1 2
3 4 5
4.002 + 35.50 - 50.00
transition point +0.00000 +0.00000
1 2
3 4 5
-50.00
invariancy interval +0.00000 +1.73913
1 2 3 4 5
-6.912 + 35.50 - 50.00
transition point +1.73913 +1.73913
2 3 4 5
1
-9.15
invariancy interval +1.73913 +3.33333
2 3 4 5
1
-3.602 + 24.00 - 40.00
transition point +3.33333 +3.33333
3 4 5
1
2
0.00
invariancy interval +3.33333
Inf
3 4 5
1 2
0.002 - 0.00 + 0.00
12
Bi-Parametric Quadratic Optimization

Primal and dual perturbed problems:

Invariancy regions instead of invariancy intervals

Illustrative example:
13
Bi-Parametric Quadratic Optimization
=40/23 =10/3
BBBBN
TBBBB
BBBBB
BBTTN
BBBBT
BBNNN
-50+35.5+0.5e2+3.5e
NBBBB
BBTTT
0
1.74
BBBBB
Invariancy regions
2
2
-50+35.5-0.78 +0.5e +3.5e
TBBBB
BBNNB
BBBTB
e=-5
NBNNN
TBTNT

3.33
-40+24-3.62
Illustrative
example
-50+35.5-6.906252

-50+35.5-3.312+0.1e2+1.2e e
NNBBB
0+0+0e+02+0e2+0e
NTBBB
BBBNB
TBBNB
TBBTB
NBBNB
NBBTB
48+20e+2.5e2+6e
NNBNB
=0
e=-8
14
Bi-Parametric Quadratic Optimization

Illustrative
example:
Optimal value
function
15
Bi-Parametric Quadratic Optimization
The
optimalregions
value function
is aare
bivariate
quadratic
Invariancy
that
transition
lines or
Invariancy
region
is
a
convex
set
and
its closure
The
optimal
boundary
value
of
a
function
non-trivial
is
continuous
invariancy
region
and
piecewise
consists
ofregions.
a is
Optimal
partition
is constant: on invariancy
function
on
invariancy
region
singletons are called trivial regions. Otherwise, they are called
a
polyhedron
that
might
be unbounded.
bivariate
finite
number
quadratic
of line
segments.
non-trivial invariancy regions.
-50+35.5-3.312+0.1e2+1.2e e
=40/23 =10/3
BBBBN
TBBBB
BBBBB
BBTTN
-50+35.5-6.906252
BBBBT
BBNNN
-50+35.5+0.5e2+3.5e
NBBBB
BBTTT
0
1.74
BBBBB
-50+35.5-0.782+0.5e2+3.5e
TBBBB
BBNNB
BBBTB
e=-5
NBNNN
TBTNT

3.33
-40+24-3.62



NNBBB
0+0+0e+02+0e2+0e
NTBBB
BBBNB
TBBNB
TBBTB
NBBNB
NBBTB
48+20e+2.5e2+6e
NNBNB
=0
e=-8
16
Bi-Parametric QO: Algorithm

Idea: reduce bi-parametric QO problem to a series of
uni-paramteric QO problems with
where
17
Bi-Parametric QO: Algorithm

Start from
Choose

Solve

, determine the optimal partition
,
and
where



Solve
where
Now, two points
and
of the invariancy region are known
Consider cases
and
on the boundary
18
Bi-Parametric QO: Algorithm

Case
19
Bi-Parametric QO: Algorithm

Case

Case

:
and
20
Bi-Parametric QO: Algorithm

Case

Case

:
and

:
and
21
Bi-Parametric QO: Algorithm

Case

Case

:
and

:
and

: back to the first or the second case
22
Bi-Parametric QO: Algorithm

Invariancy region exploration
23
Bi-Parametric QO: Algorithm

Enumerating all invariancy regions
To-be-processed queue
Completed queue
vertex
cell
edge
24
Conclusions and Future Work

•
•


•
Developed an IPM-based technique for solving bi-parametric
problems that
extends the results of the uni-parametric case
allows solving both bi-parametric linear and bi-parametric
quadratic optimization problems
systematically explores the optimal value surface
Polynomial-time algorithm in the output size
Applications in finance, IMRT, data mining

Improving the implementation

Extending methodology to
•
•
Parametric Second Order Conic Optimization
Multi-Parametric Quadratic Optimization
25
References

A. B. Berkelaar, C. Roos, and T. Terlaky. The optimal set and
optimal partition approach to linear and quadratic
programming. In Advances in Sensitivity Analysis and Parametric
Programming, T. Gal and H. J. Greenberg, eds., Kluwer, Boston,
USA, 1997.

A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Sensitivity
Analysis in Convex Quadratic Optimization: Simultaneous
Perturbation of the Objective and Right-Hand-Side Vectors.
Algorithmic Operations Research, Vol. 2(2), 2007.

A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. BiParametric Convex Quadratic Optimization. To appear in
Optimization Methods and Software, 2009.

A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. On BiParametric Programming in Quadratic Optimization.
Proceedings of EurOPT-2008, 2008.
26