LP Tricks
Some (famous) LP Modeling Tricks
(There are more … )
• Fractional LP
• Min max
• Absolute values
• Observe:
All these functions are non-linear!
1
2
Fractional …
z=
Fractional …
x
y
z=
z
3x + 2y
2y + 5x
z
†
†
x
y
x
3
y
4
1
Fractional LP
Fractional LP
•
•
•
•
T
c x +a
T
x d x+b
s.t. Ax £ b, x ≥ 0
z* := opt
Assume:
†
dT x + b > 0
cT x + a
z* := opt T
x d x+b
s.t. Ax £ b, x ≥ 0
for all feasible x
Why??
†
5
Fractional LP
†
cT x + a
z* := opt T
x d x+b
s.t. Ax £ b, x ≥ 0
†
1
dT x + b
t=
y = tx
7
y = tx
†
z* := opt {c T y + at}
y,t
s.t. Ay - bt £ 0
d T y + bt = 1
y,t ≥ 0
8
†
†
1
dT x + b
6
Fractional LP
cT x + a
z* := opt T
x d x+b
s.t. Ax £ b, x ≥ 0
• Transformation of variables:
t=
How?
We have to linearize a ratio linear function.
How do we do it?
Transformation of variables
†
†
2
Fractional LP
Fractional LP
T
c x +a
T
x d x+b
s.t. Ax £ b, x ≥ 0
z* := opt
1
t= T
d x+b
†
z* := opt {c y + at}
†
x
s.t.
y,t
s.t. Ay - bt £ 0
y = tx
d T y + bt = 1
y,t ≥ 0
Dimension: y, t ???
†
max
T
3x1 + 2x 2 + 4 x 3 + 6
3x1 + 3x 2 + 2x 3 + 5
2x1 + 3x 2 + 5x 3 ≥ 23
3x 2 + 5x 2 + 4 x 3 £ 30
x1, x 2 , x 3 ≥ 0
max 3y1 + 2y 2 + 4 y 3 + 6t
x
s.t.
9
3y1 + 3y 2 + 2y 3 + 5t = 1
†
2y1 + 3y 2 + 5 3 - 23t ≥ 0
3y1 + 5y 2 + 4 y 3 - 30t £ 0
y1, y 2 , y 3 ,t ≥ 0
10
†
†
Fractional LP
• How about ratio constraints?
Fractional Programming
• An important area of nonlinear
optimization dealing with the optimization
of functions involving ratio terms.
• There are many applications (eg
cost/benefit, DEA)
• Neglected in OR textbooks
3x1 + 2x 2 - 4 x 3
£7
5x1 + 3x 2 + 2x 3
• Is this a linear constraint?
• Can it be rewritten as a linear constraint?
†
3x1 + 2x 2 - 4 x 3 £ 7(5x1 + 3x 2 + 2x 3 )
Mind the sign of the denominator!!!
11
12
†
3
Fractional Programming
in brief
Fractional Programming
in brief
• Formats:
• Famous result:
z* := opt
x ŒX
N(x)
D(x)
z* := opt
x ŒX
k
N j (x)
j=1 D j (x)
z* := max Â
x ŒX
†
Ê N (x)
N (x) ˆ
z* := max minÁ 1 ,..., k ˜
x ŒX
Dk (x) ¯
Ë D1 (x)
z * (b ) := opt N(x) + bD(x) , b Œ ¬
x ŒX
†
There exists a b such that …
13
†
†
N(x)
D(x)
14
†
FLP Magic
Absolute me!
• |xj| appears in the objective function instead of xj.
• |aTx| £ b appears as a constraint
Action!
• Pay attention to details here!!!
Action!
15
16
4
Absolute me!
Absolute me!
y =| x |
z =| x | + | y |
y
z
†
†
x
y
x
17
18
Absolute me!
Absolute me!
• Example:
• Fact of life:
min x1 + 3 x 2 + 4 x 3
x
any real number can be expressed as the
difference of two non-negative numbers.
min x1 + 3(x 2+ + x -2 ) + 4 x 3
x
-
+
j
j
+
j
j
x j = x - x , x ,x ≥ 0
• If the sum -- rather than difference -- appears in
the objective function, then the under minimization,
the optimal
† value of at least one of the two parts
19
will be equal to zero.
If the optimal value of x2 is positive, then ??? x
will
2
†
be equal to zero.
†
+
If the optimal value of x2 is negative, then ??? x
will
2
be equal to zero.
†
†
20
5
Absolute me!
Absolute me!
• Thus, if we can force one of the two parts to be
equal to zero, …., we would have:
+
j
xj = x - x
• Example:
j
min x1 + 3 x 2 + 4 x 3
min x1 + 3(x 2+ + x -2 ) + 4 x 3
6x1 + 2x 2 + 5x 3 = 5
2x1 + 3x 2 + 4 x 3 ≥ 4
x1, x 3 ≥ 0
6x1 + 2(x 2+ - x -2 ) + 5x 3 = 5
x
x j = x +j + x -j
So, we can use this scheme if the absolute
†
value appears in the objective function.
† this remind you of something?
Does
2x1 + 3(x 2+ - x -2 ) + 4 x 3 ≥ 4
x1, x 2+ , x -2 , x 3 ≥ 0
21
22
†
†
Absolute me!
Absolute me!
• Constraints:
• How about
max x1 + 3 x 2 + 4 x 3
max x1 + 3(x 2+ + x -2 ) + 4 x 3
6x1 + 2x 2 + 5x 3 = 5
2x1 + 3x 2 + 4 x 3 ≥ 4
x1, x 3 ≥ 0
6x1 + 2(x 2+ - x -2 ) + 5x 3 = 5
3x1 - 2x 2 + x 3 £ 34
x
x
=>>
???
No!
• Rewrite it as
2x1 + 3(x 2+ - x -2 ) + 4 x 3 ≥ 4
†
x1, x 2+ , x -2 , x 3 ≥ 0
3x1 - 2x 2 + x 3 £ 34
-( 3x1 - 2x 2 + x 3 ) £ 34
Relation to non-negativity constraints?
23
†
x
24
†
†
6
Absolute me!
Absolute me!
• Such constraints appear for example in
Least Squares by linearizing the non-linear
objective function
• See next … under min max
• Constraints:
3 x1 + 2 x 2 + 4 x 3 £ 34
• What do you do now??
†
Relation to non-negativity constraints?
25
26
max min
max min
f(x)=min{2x+3,2-4x}
• Problem
max min{2x1 + 4 x 2 + 3x 3 ,3x1 + 2x 2 + 4 x 3 }
x
3x1 + 3x 2 + 7x 3 £ 8
2x1 + 2x 2 + 3x 3 £ 10
x1, x 2 , x 3 ≥ 0
• Is this an LP problem?
†
x
27
28
7
min max
min max
f(x)=max{2x+3,2-4x}
f(x)=max{2x-y,y-2x}
x
x
y
29
30
max min
min max
max min { f1 (x),..., f k (x)}
min max { f1 (x),..., f k (x)}
x ŒX
x ŒX
• Trick:
– Replace the inner (min{ }) with a new
decision variable, say t.
† – Use the new variable as a lower bound
for all the terms in min { … }
• Trick:
– Replace the inner (min{ }) with a new
decision variable, say t.
† – Use the new variable as an upper bound
for all the terms in max{…}
min t
max t
x ŒX ,t
x ŒX ,t
f1 (x) ≥ t , f 2 (x) ≥ t , ..., f k (x) ≥ t
†
f1 (x) £ t , f 2 (x) £ t , ..., f k (x) £ t
31
†
32
8
max min
Observation
• Example:
max min{2x1 + 3x 2 + 4 x 3 + 4,3x1 + 2x 2 + 5x 3 - 2}
x
• This recipe works because in either case
(maxmin or minmax), at least one of the
additional constraint is binding.
3x1 + 4 x 2 + 5x 3 £ 20
2x1 + 6x 2 + 3x 3 £ 15
x1, x 2 , x 3 ≥ 0
min max { f1 (x),..., f k (x)}
x ŒX
max t
x,t
min t
†
2x1 + 3x 2 + 4 x 3 + 4 ≥ t
†
x ŒX ,t
3x1 + 2x 2 + 5x 3 - 2 ≥ t
3x1 + 4 x 2 + 5x 3 £ 20
2x1 + 6x 2 + 3x 3 £ 15
x1, x 2 , x 3 ≥ 0
f1 (x) £ t , f 2 (x) £ t , ..., f k (x) £ t
33
†
34
†
min max
A Theorem a day keeps the doctor away!
• A very nice example of this framework is
the classical Least Square problem.
• We try to minimize the maximum deviation
of a set of points from a straight line
• Given the points, what is the “best” line?
• Theorem: Let f(x):= max{g(x),h(x)} where g and h are
convex. Then f is convex.
g(x)
h(x)
y
x
35
36
9
• Proof: on your own!
• Hint: use the definition of convexity and the definition
of max.
A Theorem a day keeps the doctor away!
• Theorem: Let f(x):= max{g(x),h(x)} where g and h are
convex. Then f is convex.
g(x)
f(x)
f (a x'+(1- a )x'') £ af (x') + (1- a ) f (x'')
"a Π[0,1]
f(x)
h(x)
†
†
y = a x'+(1- a )x''
x’
37
y
x’’
x
38
†
10