Lecture 3: Duality
1. Duality for LP in canonical form.
ˆ Duality, complementarity, and interpretations.
2. Duality for general LP problems.
3. Duality for LP in standard form
Föreläsning 5
1
Dualitet
Reading instructions for the slides:
ˆ The geometrical interpretation of the complementarity conditions
will be considered in more detail later in the course for non-linear
problems. It is probably easier to understand then, so do not spend
to much time on it here.
ˆ The economical interpretation of primal and dual problems can be
skimmed through
Föreläsning 5
2
Dualitet
Duality for LP in canonic form
Primal
Dual
maximize bT y
minimixe cT x
(P )
s.t. Ax ≥ b
x≥0
Primal feasible region
(D)
s.t. AT y ≤ c
y≥0
Dual feasible region
FP = {x ∈ Rn : Ax ≥ b; x ≥ 0} FD = {y ∈ Rm : AT y ≤ c; y ≥ 0}
x̂ ∈ FP is the optimal solution if
ŷ ∈ FD is the optimal solution if
cT x̂ ≤ cT x, ∀x ∈ FP
bT ŷ ≥ bT y, ∀y ∈ FD
Föreläsning 5
3
Dualitet
How are the primal and dual related?
Consider the following pair of primal and dual optimization problems
Primal
Dual
minimize x1 + x2
(P )
s.t.
1
x
40 1
1
x
60 1
+
+
maximize 200y1 + 400y2
1
x
50 2
1
x
50 2
≥ 200
(D)
≥ 400
x1 , x2 ≥ 0
s.t.
1
y
40 1
1
y
50 1
+
+
1
y
60 2
1
y
50 2
≤1
≤1
y1 , y2 ≥ 0
What is the relation between
ˆ the optimal values?
ˆ the objective function values (for an arbitrary feasible point) ?
ˆ the optimal solutions and the constraints at the optimum?
Föreläsning 5
4
Dualitet
Illustration of the feasible region, iso-cost lines, and the optimal solution
x2
60
c
20000
y2
b
ŷ
50
x̂
40
FP
30
10000
20
10
8000
16000
24000 x1
FD
10
20 30
40 50
y1
ˆ The Optimal values are the same cT x̂ = bT ŷ = 20000
ˆ For each x ∈ FP and y ∈ FD it holds that
cT x ≥ cT x̂ = bT ŷ ≥ bT y
Föreläsning 5
5
Dualitet
Let
 
 
s1
200




ŝ =
= Ax̂ − b =
s2
0
 
 
1
r1
r̂ =   = c − AT ŷ =  6 
0
r2
(S)
(R)
The optimal solution satisfies
1. ŝ = Ax̂ − b ≥ 0, x̂ ≥ 0 (primal feasibility)
2. r̂ = c − AT ŷ ≥ 0, ŷ ≥ 0 (dual feasibility)
3. xj rj = 0, j = 1, 2 and yi si = 0, i = 1, 2 (complementarity)
Föreläsning 5
6
Dualitet
The complementarity conditions can be given a geometrical
interpretation. From (R) and (S)
 
   
 
   
1
1
200
1
1
1 1
60 
50 









+
50
(S)
=
20000 −  200
(R)
=
1
1
400
0
1
0 6
50
50
it follows that the gradients of the objective functions (c and b
respectively) can be given as linear combinations of the active
constraints. This is illustrated in the figure. (Note: incorrect scale)
x2
c
y2
b
FP
FD
x1
Föreläsning 5
y1
7
Dualitet
Economical interpretation
ˆ We consider the production of two products P1 and P2
ˆ Machines M1 and M2 are used for the production of P1 and P2
ˆ The Production capacities for the machines are [day/unit]:
M1
1
40
1
60
M2
1
50
1
50
P1
P2
ˆ The price for the products [SEK/unit]:
product
price
P1
b1 = 200
P2
b2 = 400
ˆ In the dual, production volume is determined for profitmaximization.
Föreläsning 5
8
Dualitet
Assume that you have the option of renting out the machines M1 and
M2 . What is the lowest possible price xk SEK/day for renting out Mk ,
k = 1, 2.
The lowest price is determined from the following optimization problem
minimize x1 + x2
s.t.
1
x
40 1
1
x
60 1
+
+
1
x
50 2
1
x
50 2
≥ 200
≥ 400
x1 , x2 ≥ 0
where the constraints ensure that you earn at least as much by renting
out the machines as the profit you would have made producing products
P1 and P2 .
Föreläsning 5
9
Dualitet
ˆ The primal and dual can be seen as alternatives to each others.
– If your business idea is to rent out M1 and M2 , then the dual
provides information about the lowest production volume
necessary for instead using the machines for producing P1 and
P2 .
– if your business idea is to produce P1 and P2 , then the primal
provides information of the lowest renting price that makes it
profitable to rent out the machines instead. (break-even price)
ˆ What is called primal and dual is actually arbitrary, since it can be
shown that the dual of the dual is the primal.
(at least for linear programs)
Föreläsning 5
10
Dualitet
Theoretical results
The following result from the book (Griva, Nash & Sofer) confirms the
relations in the example:
Theorem 6.4[Weak duality]
For every x ∈ FP and every y ∈ FD it holds that cT x ≥ bT y.
A direct implication from this is that, if x̂ ∈ FP and ŷ ∈ FD and
cT x̂ = bT ŷ, then x̂ and ŷ are optimal to (P ) and (D), respectively.
Föreläsning 5
11
Dualitet
Theorem 6.9[Strong duality - “stronger than the book”]
There are four alternative for the optimization problems (P ) and (D)
(a) FP 6= ∅, Fd 6= ∅. Then there is (at least) one optimal solution x̂ to
(P ) and (at least) one optimal solution to (D). They satisfy
cT x̂ = bT ŷ.
(b) FP 6= ∅ but FD = ∅. Then the primal is unbounded (from below).
(c) FD 6= ∅ but FP = ∅. Then the dual is unbounded (from above).
(d) FP = ∅ and FD = ∅.
ˆ The proof of (a) is non-trivial and is usually based on Farkas’ lemma.
ˆ Case (b) can be interpreted as a consequence of the lack of a lower
limit for the primal due to the lack of solutions to the dual.
Similar interpretation holds for (c).
Föreläsning 5
12
Dualitet
Notation




a11 . . . a1n
āT1
h
i  
 .

.. 
.. 

..
A=
=
=
â1 . . . ân
. 
 . 

āTm
am1 . . . amn
 
 
 
s1
y1
b1
 . 
 . 
 . 




..  ,
.
.
b =  . , y =  . , s = 
 
sm
bm
ym
 
 
 
r1
x1
c1
.
.
.
..  , x =  ..  , r =  .. 
c=
 
 
 
rn
xn
cn
Föreläsning 5
13
Dualitet
Theorem 6.12 [Complementary slackness]
x ∈ Rn is optimal for (P ) and y ∈ Rm is optimal for (D) if and only if
xj ≥ 0, rj ≥ 0, xj rj = 0,
yi ≥ 0, si ≥ 0, yi si = 0,
j = 1, . . . , n,
i = 1, . . . , m,
where s = Ax − b och r = c − AT y.
Terminology:
ˆ Non-negativity constraints for the primal: xj ≥ 0, j = 1, . . . , n.
ˆ General constraints for the primal: si = āTi x − bi ≥ 0, i = 1, . . . , m.
ˆ Non-negativity constraints for the dual: yi ≥ 0, i = 1, . . . , m.
ˆ General constraints for the dual: rj = cj − âTj y ≥ 0, j = 1, . . . , n.
Föreläsning 5
14
Dualitet
A consequence of the complementarity Theorem is that
ˆ if rj = cj − âTj y > 0 then xj = 0,
i.e., the j:th non-negativity constraint of the primal is active.
ˆ if yi > 0 then si = āTi x − bi = 0,
i.e., the i:th general constraint of the primal is active.
ˆ if si = āTi x − bi > 0 then yi = 0,
i.e., the i:th non-negativity constraint of the dual is active.
ˆ if xj > 0 then rj = cj − âTj y = 0,
i.e., the j:th general constraint of the dual is active.
Föreläsning 5
15
Dualitet
Geometric interpretation of the complementarity constraints
h
iT
Let ej = 0 . . . 0 1 0 . . . 0 ∈ Rn , where the 1 is in position j.
Then
X
X
T
ej rj −
āi yi
c=r−A y =
j:rj >0
i:yi >0
i.e., the gradient of the primal objective function is a linear combination
(with coefficients > 0) of the normal vectors of the active constraints.
h
iT
Let ei = 0 . . . 0 1 0 . . . 0 ∈ Rm , where the 1 is in position i.
Then
X
X
b = Ax − s =
âj xj −
ei si
j:xj >0
i:si >0
i.e., the gradient of the dual objective function is a linear combination of
the normal vectors of the active constraints.
Föreläsning 5
16
Dualitet
Graphic illustration of the geometrical interpretation
â1
b
â2
c
FP
ā2
ā1
FD
ˆ c can be written as a linear combination of ā1 and ā2
ˆ b can be written as a linear combination of â1 and â2
Föreläsning 5
17
Dualitet
Duality for general LP-problems
Primal
minimize cT1 x1 + cT2 x2
Dual
maximize bT1 y1 + bT2 y2
s.t. A11 x1 + A12 x2 ≥ b1
s.t. AT11 y1 + AT21 y2 ≤ c1
A21 x1 + A22 x2 = b2
AT12 y1 + AT22 y2 = c2
x1 ≥ 0, x2 free
y1 ≥ 0, y2 free
Two methods for deriving the dual
ˆ Reformulate the problem in canonical form, for which the dual is
know.
ˆ By using Lagrange-relaxation (more about this later in the course)
Föreläsning 5
18
Dualitet
Duality for LP in standard form
Primal
Dual
minimize cT x
(P )
maximize bT x
(D)
s.t. Ax = b
s.t. AT y ≤ c
x≥0
FP = {x ∈ Rn : Ax = b; x ≥ 0}
FD = {y ∈ Rm : AT y ≤ c}
Weak duality: x ∈ FP and y ∈ FD
cT x − bT y = cT x − yT Ax = (c − AT y)T x ≥ 0.
Complementarity: x ∈ FP and y ∈ FD is optimal for (P ) and (D)
respectively, if and only if (c − AT y)T x = 0, i.e. if (ci − âTi y)xi = 0
Föreläsning 5
19
Dualitet
A consequence of complementarity
Assume that we with the simplex method determined an optimal solution
of the primal (P ). This means that we have basic- and non-basic
variables with indecies β = (β1 , . . . , βm ) and η = (η1 , . . . , ηl ) so that

−1

x
=
B
b

B



x = 0
N
T

B
y = cB





ĉN = cN − NT y ≥ 0
⇒

T

B

 y = cB
NT y ≤ cN


 T
c x = cTB xB = bT y
⇒

AT y ≤ c
cT x = bT y
The last implication means that y is optimal for (D).
Föreläsning 5
20
Dualitet
Conversely, assume that we have found an optimal solution y to the dual
(D). We can construct an optimal solution x to the primal by using the
complementarity constraints:
(ci − âTi y) · xi = 0,
i = 1, . . . , n
If ci − âTi y > 0 it holds that xi = 0, i.e. i ∈ η (non-basic variable). If
there are n − m such non-active dual constraints, then we have found a
set of basic variable indecies β for which the optimal x is given by
xB = B−1 b and xN = 0
If it is easy to solve the dual, this is a good alternative. This is usually
the case if n >> m.
Föreläsning 5
21
Dualitet
Example
minimize x1 + 2x2 + 3x3 + 4x4 + 5x5
s.t. 5x1 + 4x2 + 3x3 + 2x2 + x1 = 1
xk ≥ 0, k = 1, . . . , 5
The dual is
maximize y
s.t. 5y ≤ 1, 4y ≤ 2, 3y ≤ 3, 2y ≤ 4, y ≤ 5
which has the optimal solution ŷ = 51 . From the complementarity
constraint it is clear that x̂2 = x̂3 = x̂4 = x̂5 = 0 and for the primal
constraint to be satisfied x̂1 = 15 . This is the optimal solution to the
primal.
Föreläsning 5
22
Dualitet
Reading instructions
ˆ Chapter 6.1-6.2 in the book.
Föreläsning 5
23
Dualitet