Optimization with piecewise-affine cost
functions
G. Ferrari Trecate, Paolo Letizia, Matteo Spedicato
Automatic Control Laboratory,
Swiss Federal Institute of Technology (ETH),
Zurich, Switzerland
21 June 2001 - Technical Report AUT01-13
Abstract
This report shows how it is possible to solve the problem of the minimization
of an objective function composed by the sum of piece-wise affine functions
as a Mixed Integer Linear Programming (MILP) problem.
The optimization problem
m
P
min
x
i=1
ki ∗ pi (xi )
subject to
Ax ≤ B



x=
,

x1
.
.
xm





where pi (xi ) are piece-wise affine functions and A, B are the constraint matrices, will be reformulated as a MILP problem:
min
KV
V
subject to F V ≤ G
where
K=
h
k1
.
.
kv
i
,


V =


v1
.
.
vv





and F , G are the constraint matrices.
In the first chapter of this report a simple case (cost function composed
of only one piece-wise affine function) will be considered. The generalization
to the case of a cost function composed of m piece-wise affine functions will
be contained in the second chapter.
Chapter 1
Simple case
Consider the following optimization problem:
min
x
subject to
k ∗ p(x)
Ax ≤ B
(1.1)
where k is a scalar coefficient, p(x) is a piece-wise affine function defined on
the real variable x, A and B are suitable column vectors which define the
feasible region for x. The piece-wise affine function p(x) is composed by n
affne functions, each one defined on a different domain:


a1 x + b 1





 a2 x + b 2
p(x) = 






if
if
.
.
an x + b n
if
x ∈ [l1 , u1 ]
x ∈]u1 , u2 ]
.
.
x ∈]un−1 , un ]
(1.2)
where the lower bound l1 and the upper bound un of the variable x are defined
by the inequalities Ax ≤ B. Note that p(x) could have jump discontinuity
for x = ui , i = 1, ..., n − 1, as shown in the Figure 1.1 assuming n = 3.
We are interested to reformulate the optimization problem (1.1) as a
Mixed Integer Linear Programming (MILP) problem:
min
KV
V
subject to F V ≤ G
(1.3)
where K is the weights vector; V is the optimization variables (both continuous and logic) vector; F and G are the matrices which define the constraints
of the vector V . Note that:
1
Figure 1.1: Example of a piece-wise affine function with equation (2) assuming
n = 3.
• K will contain the coefficient k;
• V will contain the variable x;
• F and G will contain the matrices A and B,
where k, x, A and B are the same defined in (1.1).
To derive a MILP problem we need to introduce some auxiliary logic and
real variables.
First of all introduce n − 1 binary variables, each one defined in the
following way:
δ 1 = 1 ⇔ x ≥ u1
δ 2 = 1 ⇔ x ≥ u2
.
(1.4)
.
δ n−1 = 1 ⇔ x ≥ un−1
2
So, by giving a binary value to the variables δ i , i = 1, ...., n−1, it is possible to
choose a particular region for the variable x. For example, for the piecewiseaffine function reported in Figure 1.1 we will introduce two binary variables
defined as below:
δ1 = 1
δ2 = 1
⇔
⇔
x≥4
x ≥ 12.
and assuming for instance (δ1 = 1) ∧ (δ2 = 0), the region 4 ≤ x < 12 is
defined.
Note that the variables δ i , i = 1, ..., n − 1, are related by the logical
equations
δi = 1
=⇒
δj = 1
∀j < i, i = 2, ...., n − 1
(1.5)
which imply that not all the combinations of the variables δ i , i = 1, ...., n − 1,
are allowed. Note also that the number of the relations (1.5) is equal to
(n − 2) + (n − 3) + ... + 1.
Define now n − 1 real variables z as below:
z1 =
z2 =
zn−1 =
(
(
.
.(
if δ 1 = 1
else
a2 x + b2
a 1 x + b1
(1.6)
if δ 2 = 1
else
(a3 − a2 )x + (b3 − b2 )
0
(1.7)
if δ n−1 = 1
else
(an − an−1 )x + (bn − bn−1 )
0
(1.8)
and define the real variable ze given by the sum of the variables zi , i =
1, ....., n − 1:
ze = z1 + z2 + ... + zn−1
(1.9)
It’s easy to verify that, depending on the feasible combination of the
variables δ i , i = 1, ...., n − 1, ze will be equal to one of the n pieces of the
function (1.2).
3
1.1
Inequalities
The definition of the binary variables δ (1.4) and their relations (1.5), of the
real variables z (1.6 - 1.8) and ze (1.9) can be translated into mixed integer
linear inequalities, as shown in [1], [2] and reported in Table 1.1, 1.2.
(1.4)
(1.5)
inequalities
(
D1 δ 1 ≤ −(u1 − x) + D1
(d1 − ε)δ 1 ≤ (u1 − x) − ε
(
D2 δ 2 ≤ −(u2 − x) + D2
(d2 − ε)δ 2 ≤ (u2 − x) − ε
.
.
(
Dn−1 δ n−1 ≤ −(un−1 − x) + Dn−1
(dn−1 − ε)δ n−1 ≤ (un−1 − x) − ε
δ i − δ j ≤ 0 ∀j < i, i = 2, ...., n − 1
no ineq.
2 ∗ (n − 1)
(n − 2) + (n − 3) + ... + 1
Table 1.1: Translation in inequalities of the definition of the variables δ and
of their logical relations.
In the Table 1.1 ε is a small tolerance (typically the machine precision); Di
and di , i = 1, ..., n−1, represent respectively the maximum and the minimum
of (ui − x) , where x ∈ [l1 , un ]. Theoretically, an over[under] estimate of the
maximum[minimum] suffices for our purpose, so that one can assume
∀i = 1, ..., n − 1
Di =max (ui − x) = max(ui ) − min(x) = un − l1
∀i = 1, ..., n − 1
di = min(ui − x) = min(ui ) − max(x) = u1 − un .
i
In the Table 1.2
• M1 , m1 are respectively the maximum and the minimum of a1 x + b1 ,
where x ∈ [l1 , un ] ;
• M2 , m2 are respectively the maximum and the minimum of a2 x + b2 ,
where x ∈ [l1 , un ] ;
• Mj , mj , j = 3, ..., n, are respectively the maximum and the minimum
of (aj − aj−1 )x + (bj − bj−1 ), where x ∈ [l1 , un ] ;
4
(1.6)
(1.7)
(1.8)
(1.9)
inequalities

(M2 − m1 )δ 1 − z1 ≤ −a2 x + (M2 − m1 − b2 )



 (M − m )δ + z ≤ a x + (M − m + b )
1
2 1
1
2
1
2
2


 (m2 − M1 )δ 1 − z1 ≤ −a1 x − b1

(m1 − M2 )δ 1 + z1 ≤ a1 x + b1


 −M3 δ 2 + z2 ≤ 0

 m δ −z ≤0
3 2
2

−m
δ
+
z2 ≤ [(a3 − a2 )x + (b3 − b2 )] − m3
3
2



M3 δ 2 − z2 ≤ −[(a3 − a2 )x + (b3 − b2 )] + M3
.
.

−M
δ
+
z
≤
0

n n−1
n−1


 m δ
n n−1 − zn−1 ≤ 0

−mn δ n−1 + zn−1 ≤ [(an − an−1 )x + (bn − bn−1 )] − mn



Mn δ n−1 − zn−1 ≤ −[(an − an−1 )x + (bn − bn−1 )] + Mn
(
ze ≤ z1 + z2 + ... + zn−1
ze ≥ z1 + z2 + ... + zn−1
no ineq.
4 ∗ (n − 1)
2
Table 1.2: Translation in inequalities of the definition of the variables z and
z̃.
• ε is once more the machine precision.
As explained before, also in this case an over[under] estimate of the maximum[minimum] suffices for our purpose, so that one can assume a global
maximum M = max(Mi ) and a global minimum m = min(mi ), i = 1, ..., n
and replace all the Mi , mi , i = 1, ..., n listed before with them.
1.2
Derivation of the MILP problem
It is now possible to reformulate the initial problem
min
x
subject to
as the MILP problem
5
k ∗ p(x)
Ax ≤ B
min
KV
V
subject to F V ≤ G
where:

ze
z1
.
.








 zn−1
V =
 x


 δ1

 .


 .
δ n−1
h










;









i
K = k 0 . . 0 , where the number of 0 is equal to 2(n − 1) + 1);
F and G are matrices of suitable sizes such that they specify the inequalities Ax ≤ B and those contained in Tables 1.1, 1.2.
6
Chapter 2
Generalization
Based on the simple case explained before, it’s easy to generalize the results
to the case of a cost function composed of m piece-wise affine functions. The
extended optimization problem will be:
m
P
min
x
i=1
subject to
Ax ≤ B
ki ∗ pi (xi )
,



x=

x1
.
.
xm





(2.1)
where ki , i = 1, ..., m, are scalar coefficients, pi (xi ), i = 1, ..., m, are piecewise affine functions each one defined on the real variable xi , A and B are
suitable matrices which define the feasible region for the real variables xi ,
i = 1, ...m. The ith piece-wise affine function pi (xi ) is composed by ni affine
functions:


ai,1 xi + bi,1





 ai,2 xi + bi,2
pi (xi ) = 






if
if
.
.
ai,ni xi + bi,n if
xi ∈ [li,1 , ui,1 ]
xi ∈]ui,1 , ui,2 ]
.
.
xi ∈]ui,n−1 , ui,n ]
, i = 1, ..., m.
(2.2)
Our aim is once more to obtain a MILP formulation for the problem (2.1).
For this purpose we need to introduce auxiliary logic and real variables. In
particular, analogously to what has just been explained for the simple case,
ni − 1 logic and ni − 1 real variables are needed for each pi (xi ).
7
Define the logic variables δ as follow:
δ i,1 = 1 ⇔ xi ≥ ui,1
δ i,2 = 1 ⇔ xi ≥ ui,2
.
.
δ i,ni −1 = 1 ⇔ xi ≥ ui,ni −1
, i = 1, ..., m
(2.3)
The logic variables δ i,j , i = 1, ..., m, j = 1, ..., ni − 1 are related by the
following logical equations:
δ i,j = 1 =⇒ δ i,s = 1 ∀s < j, j = 2, ...., ni − 1
i = 1, ..., m
(2.4)
The same conclusions of the simple case are valid ∀i, i = 1, ..., m.
Define now the real variables z as follows:
zi,1 =
zi,2 =
zi,ni −1 =
(
(
.
.(
if δ i,1 = 1
else
ai,2 xi + bi,2
ai,1 xi + bi,1
(2.5)
if δ i,2 = 1
else
(ai,3 − ai,2 )xi + (bi,3 − bi,2 )
0
(2.6)
if δ i,ni −1 = 1
else
(ai,ni − ai,ni −1 )xi + (bi,ni − bi,ni −1 )
(2.7)
0
and define the real variable zei given by the sum of the variables zi,j , j =
1, ....., ni − 1:
zei = zi,1 + zi,2 + ... + zi,ni −1 .
(2.8)
2.1
Inequalities
Analogously to the simple case the definition of the binary variables δ (2.3)
and their relations (2.4), of the real variables z (2.5 - 2.7) and ze (2.8) can be
translated into mixed integer linear inequalities. The results of this procedure
are shown in the Tables 2.1, 2.2.
8
(2.3)
(2.4)
inequalities
(
Di,1 δ i,1 ≤ −(ui,1 − xi ) + Di,1
(di,1 − ε)δ i,1 ≤ (ui,1 − xi ) − ε
(
Di,2 δ i,2 ≤ −(ui,2 − xi ) + Di,2
(di,2 − ε)δ i,2 ≤ (ui,2 − xi ) − ε
.
.
(
Di,ni −1 δ i,ni −1 ≤ −(ui,ni −1 − xi ) + Di,ni −1
(di,ni −1 − ε)δ i,ni −1 ≤ (ui,ni −1 − xi ) − ε
δ i,j − δ i,s ≤ 0 ∀s < j, j = 2, ...., ni − 1
no ineq.
2 ∗ (ni − 1)
(ni − 2) + (ni − 3) + ... + 1
Table 2.1: Translation in inequalities of the definition of the variables δ and
of their logical relations.
(2.5)
(2.6)
(2.7)
(2.8)
inequalities


 (Mi,2 − mi,1 )δ i,1 − zi,1 ≤ −ai,2 xi + (Mi,2 − mi,1 − bi,2 )

 (M − m )δ + z ≤ a x + (M − m + b )
i,1
i,2 i,1
i,1
i,2 i
i,1
i,2
i,2

(m
−
M
)δ
−
z
≤
−a
x
−
b
i,2
i,1
i,1
i,1
i,1
i
i,1



(mi,1 − Mi,2 )δ i,1 + zi,1 ≤ ai,1 xi + bi,1

−Mi,3 δ i,2 + zi,2 ≤ 0



 m δ −z ≤0
i,3 i,2
i,2

−m
δ
+
zi,2 ≤ [(ai,3 − ai,2 )xi + (bi,3 − bi,2 )] − mi,3
i,3 i,2



Mi,3 δ i,2 − zi,2 ≤ −[(ai,3 − ai,2 )xi + (bi,3 − bi,2 )] + Mi,3
.
.


 −Mi,ni δ i,ni −1 + zi,ni −1 ≤ 0

 m δ
i,ni i,ni −1 − zi,ni −1 ≤ 0

−mi,ni δ i,ni −1 + zi,ni −1 ≤ [(ai,ni − ai,ni −1 )xi + (bi,ni − bi,ni −1 )] − mi,ni



Mi,ni δ i,ni −1 − zi,ni −1 ≤ −[(ai,ni − ai,ni −1 )xi + (bi,ni − bi,ni −1 ) + Mi,ni
zei ≤ zi,1 + zi,2 + ... + zi,ni −1
zei ≥ zi,1 + zi,2 + ... + zi,ni −1
Table 2.2: Translation in inequalities of the definition of the variables z and
z̃.
9
Fixed i, the number of inequalities which translate the equations (2.5-2.7)
is 4 ∗ (ni − 1), and the number of inequalities which translate the equations
(2.8) is 2.
Moreover, fixed i, the definitions and the considerations about the coefficients ε, M, m, D, d are the same reported for the simple case.
2.2
Derivation of the MILP problem
It is now possible to reformulate the initial problem
min
x
subject to
m
P
i=1
ki ∗ pi (xi )
Ax ≤ B
,
KV
V
subject to F V ≤ G
where:
V =
"
0
V
00
V
#
,
10



x=
as the MILP problem
min

x1
.
.
xm






zf1
.
.
zf
m
z1,1
.
.















 z1,n −1
1


z2,1
0

V =

.


.


 z2,n2 −1


.


.


 zm,1


.



.
zm,nm −1
K =
h




















;



















V
00
x1
.
.
xm
δ 1,1
.
.















 δ 1,n −1
1


δ 2,1

=

.


.


 δ 2,n2 −1


.


.


 δ m,1


.



.
δ m,nm −1
i




















;


















k1 . . km 0 . . 0 , where the number of 0 is equal to
m
P
i=1
2(ni − 1) + 1;
F and G are matrices of suitable sizes such that they specify the inequalities Ax ≤ B and those contained in the tables 2.1 and 2.2.
2.3
References
[1] Bemporad, A., & Morari, M. Control of systems integrating logic, dynamics, and constraints. Automatica, 35(3): 407-427, 1999.
[2] Mignone, D. The Really Big Collection. Tecnical Report - IfA, 2000.
11